The Entac Tech Blog

I’ve had enough. I’m sick of trying to secure the network from enterprise endpoints that are catastrophically vulnerable to the latest digital malady and are a complete pain in the arse to manage. Enterprise software packaging, group policy, anti-virus and anti-spyware: I just don’t want to deal with it any more.

Whatever Microsoft has been doing for the past four years or so to make all this management of desktops easier and not make sysadmins want to weep with despair and rage into their cornflakes every morning, it’s not enough.   Not least because every time they come up with a centralised management platform it means I have to poke more holes through my security tools to let them manage the damn things. Endpoint security companies can fuck off too.  Seriously – we’re still relying on signatures in the 21st century?  Really?  Enough is enough.

Wild West Networking
Here’s what I’m going to do. I’m going to give my users pervasive wireless – apart from anything else, they’re desperate to have it anyway. I’m going to stop telling them what they can and can’t connect to the network. At this stage all that happens is folk go and connect some fruity piece of consumerware anyway and create more headaches. In fact I want to use my Macbook on the network too! Why should the users get all the fun?

I’m going to pull the whole security perimeter back to the DC and let them fight it out on the LAN. I may even let someone else run that LAN for me – from now on it’s just a remote access technology. The LAN has become an un-trustable nightmare, and it’s time that the security architecture reflected that. In the meantime, I enable BYOD and VDI on the network and all the execs become my friends.  Lucky me.

Aligning on Identity
Everything becomes a remote access technology – LAN, wireless, LTE, 3G, hotspots, home broadband – everything. Now that location and IP address aren’t a good marker for what access someone should get, what can I use? Well, the user still has to authenticate, and to do that, they have an identity on the network. Cisco have been talking a lot recently about identity based policy enforcement, and I agree with them. The same user should get the right policy, no matter the device, location or access technology they use.

So, our core platform becomes the user’s identity. We then configure the relevant policies and attach them to the user’s endpoint as they come on to the network. We disable direct access to data and make them to use a VDI session to work. This means that they have to be connected to the network to do anything, but that’s generally a good thing. Most organisations are terrified about data leakage – those that know what it is anyway – and this is as good a way as any to stop that whole nightmare. It’s not a panacea: after all, everyone has cameras on them all the time now, but technology can only do so much. Policy is equally or more important.

Now that the LAN is untrusted I can actually get rid of those expensive desktops that don’t work very well, and need lots of management and TLC just so that employees can do their bloody jobs. I also don’t have to spend a tonne of cash on helpdesk stuff just because desktops are too easy to screw up. I, for one, would be delighted to never see a Windows desktop on an enterprise network ever again.

Security Landscape
To support this kind of change, there’s a need for some re-tooling. Apart from anything else, the perimeter needs to be application and identity aware so that it’s not a pain to manage. I’m going to need better orchestration tools for security functions so that it doesn’t take me weeks to build new services, and I’ll need some good data sources to help me decide on where to deploy new tools and resources, as well as to give me an insight into what’s happening on the network.

I now declare that the core technologies for network security are going to be:
- Identity
- Virtualisation (of everything)
- VPN
- Application aware firewalls
- Orchestration
- Correlation, analytics and profiling

That All Sounds a Bit Scary – What’s the Alternative
Orbit. Nuke. It’s the only way to be sure.

–nta

A number of my current clients are exploring the possibilities of persistent connectivity through the implementation of wireless, along with the dreaded BYoD prospect. BYoD is an attractive concept for senior managers, because they often think of it in terms on saving money on hardware, software, and management costs. After all, why would you supply an employee with a smartphone, tablet, and laptop when, as Apple knows, it’s easy enough to persuade them to buy their own. I’ll leave it up to you whether you believe that BYoD will actually cost or save money.

Now before we dive into the requirements for BYoD in an enterprise context, let’s just think about the infrastructure required to support that choice.

Policy Headaches
First up, you should consider the policy aspects of the BYoD prospect. At this stage you want to try and define what users will be allowed to do and how you’re going to monitor and enforce that policy. This can range from only allowing a hybrid mobile/remote access service, to full identity based differentiated access to the full gamut of your enterprise services.

Secondly, you need to consider the security implications of mobile devices on the network, and the possibility of data being stored on devices. For most BYoD users, email and document editing are the killer apps. Now, I don’t know about you, but virtual desktop services like Citrix XenApp really suck when used on any sort of touchscreen device. There’s a good reason why the Microsoft tablet PC died a slow death a few years ago. To be effective then, the average BYoD user is going to want access to their email and document shares. That creates a headache in terms of the data then cached or actively saved on the device.

There are two broad solutions to this:

1. Mobile Device Management: get your users to sign up to giving you control over their device so that you can wipe it in the event that the device is lost or stolen. This is usually fine for corporate issued devices, but people get twitchy when you tell them that their iPhone will get nuked from orbit if they leave the company.

2. Containerisation: this is a relatively new idea that requires the device to support the creation of a virtual partition for the storage of corporate data and apps. This approach is generally more palatable, but is patchy in terms of platform support.

Ideally, we’d just assume users will be connected to the network somehow, and simply push a virtual mobile device at them as an app and wipe that if anything untoward happened to the device, but as yet, I’ve not heard of such a solution.

The Support Nightmare
Once all of this is in place, you need to think about the specifics of BYoD. At a minimum, you’re going to want to ask yourself the following questions:

1.What devices are you going to support – everything? Just iDevices? Android too? Will anyone buy a WinPhone?

2. What access should users get from the BYoD network? What services will they use? How will you segment BYoD devices from the rest of the corporate network?

3. Is your solution going to support just data, or voice and video too?

4. What process should your users go through when connecting their device to the network? What’s the “on-boarding” process?

6. How will you handle the billing of data plans and other mobile connectivity bills for non-corporate devices?

7. What policies will you enforce with regard to data security on non-coroporate devices?

To support this, you need persistent connectivity, whether that’s in the office, or on the move. Given the mobility of the modern enterprise workforce, connectivity requirements no longer stop at the office door. Persistent connectivity in this context means a mix of wired, wireless, and mobile. Not only must your infrastructure support all of these connection methods inside the enterprise perimeter (or what’s left of it), you also need to be able to support a range of devices connecting to the enterprise network remotely from a wide range of different connectivity scenarios and in a secure manner. Worse, your network admins are going to end up on the hook for the connectivity experience of users who really don’t care about the distinction between GPRS, EDGE, 3G, LTE (LOL!), wifi and home broadband. And with persistent presence apps making a splash into the smart device market, that problem is also going to affect the remote users’ voice and video experience.

Network connectivity is only fully under the control of the enterprise inside the perimeter, i.e. in offices and datacentres. Within this space, you need to be looking at the following technologies:

- Pervasive enterprise class wireless

- Presence solutions such as Cisco Jabber

- Identity based wired and wireless networking

- PKI

- Secure remote access

- Mobile device management

Lies, Damn Lies, and Economics
Taking all this into account when discussing the financial benefits of BYoD and persistent connectivity is vital. Senior management may expect an almost immediate reduction in costs, however this is unrealistic. It’s important to bear in mind that the cost reductions possible through BYoD will take two to three years to be realised, and will require substantial investment in years one and two to achieve.

I think it’s fair to say that the economic benefits of BYoD have been somewhat exaggerated, but don’t be fooled by that. The people who want to use their expensive Apple kit at work are generally the type of folk to whom “policy” and “compliance” are optional. By appealing to their cost-driven mentality you can keep the BYoD and mobility conversations sane. And potentially end up with some fun new systems to play with.

As part of my current assignment, I’ve been asked to come up with a design for a complete refresh of the client’s network.  This is a fairly large public sector client, and as a result I was keen to see how the existing network is managed.  Given the title of the post, I’m sure you won’t be surprised to hear that the current management architecture is a bit hit and miss.  The server estate is very well managed, and there are a significant number of management tools deployed, with SCCM on the way too (although, as an aside, I observe that the Microsoft website is very light on information about what it can actually do).

Management is Out of Scope?

Management of the network side of things is not so good, however.  When I arrived onsite, a number of switches and other network devices had no management addresses, and seemed to have been configured to run everything on VLAN 1.  The switches that had management addresses were configured on a different VLAN from most of the management tools.  When I asked where their syslog server was so I could help troubleshoot a fault, I got blank looks.  When, as part of the design requirements gathering, I mentioned the need for a management zone, I was told it was out of scope.

Cue puzzlement.  I know that for the majority of IT professionals the network’s just there.  They pay management of it no more mind than the average person thinks about capacity planning on the rail network.  But to take this head in the sand approach ignores a couple of very important factors:

1. The network is an absolutely critical resource for all IT organisations. Without a network, there’s no way for your users/clients to connect to all the fancy servers.

2. A properly built network management infrastructure reduces the time to fix for faults and is invaluable in the fault diagnosis process.  This applies to server and end client faults as well as faults on the network itself.

3. Network devices contain a tonne of data about what’s going on on your network.  Without this data, you are flying blind as to what’s on your network and what it’s doing.

It’s something that seems so obvious to me, that I’m always surprised when asked to justify management tool costs.  Organisations are quite happy to spend hundreds of thousand of pounds on a well-known OS vendor, buying server licences by the dozen, but there’s never any room for management tools that cost a couple of k for an enterprise licence.  A lot of tools can be put in place for nothing, and simply need to be enabled on your network gear..  Syslog, for example, is built-in to most Linux server distros and is pretty easy to set up.  Point all your network deivces at it and at least you can be sure that the logs explaining why a switch rebooted won’t be wiped by the reboot!

8 Ways to Improve Your Network Management Zone

So, what should you do about it?  Here are a few tips:

1. Enable syslog on your network devices and point them at a central server.

2. If you have budget for management tools, look into a Splunk enterprise licence.  Ignore the name, Splunk has saved me hours trawling through multiple device logs looking for problems.  To make the business case to your management, tell them they can get pretty reports out of it too – it’ll take five minutes to set up.  If you’re indexing less than 500Mb of data a day, you can use Splunk for free.

3. Put everything on a management network and use dedicated management ports where available.  And for pity’s sake, don’t leave your external firewalls with a public IP address assigned to the management interface.

4. Put a firewall between the management network and the rest of the LAN. It doesn’t have to be a separate firewall, just trunk a VLAN up to your existing firewall.  If you want to force admins to VPN into the management LAN, it certainly wouldn’t hurt.

5. Set up AAA on everything that supports it and use a RADIUS or TACACS server on your management LAN to keep track of who’s doing what. FreeRADIUS, Cisco ACS, or (if you must) IAS are options for this.

6. Disable insecure management protocols wherever you can.  It’s a mystery to me why anyone thinks telnet is an acceptable way to connect to anything these days, but some devices don’t support anything better.  Lock it down as best you can.

7. Enable NetFlow on everything that supports it and point it at a sink on the management LAN.  You can use free tools such as the nfdump suite to analyse the data.  NFsen will also draw pretty graphs from the same data.  If you’re concerned about the data being carried by NetFlow, encrypt the traffic with IPSec.

8. SNMP is a great tool for monitoring all sorts of things, including network availability and topology changes.  Enable it, point it at a central SNMP client.

Pretty much everything I’ve mentioned there can be done using a single Linux server VM (although high data volume Splunk indexers may need dedicated resources), and the vast majority of the products mentioned can be used for free.

So what are you waiting for?  Spin up your favourite Linux server distro on the hypervisor of your choice and have fun seeing what’s happening on your network.  And enable syslog on the hypervisor while you’re there!

–nta

I know, it seems unlikely – but this is really good. Check it out.

http://lamejournal.com/2011/12/07/guide-to-making-changes/

–nta

As I make my way home through Hurricane Bawbag, I find myself thinking about the continued relevance of firewalls.  For many years now, a perimeter firewall has been considered to be the aboslute mimimum level of protection required to defend networks from Internet-borne threats, so much so that they are even built into end users’ broadband routers.  The ubiquity of the modern firewall is almost as broad as Internet connectivity (yes, there _are_ people out there who run without firewalls!).  While overall this is a good thing, it’s becoming increasingly clear that firewalls are becoming less effective as a security tool, and may even be causing unnecessary work for network and system admins.  It’s got to the stage that rulebases are so large and complex that some organisations need dedicated teams of people to look after firewalls!

I would argue that, in general, that a requirement for a team of people to look after a well-understood device type is an indication of failure both in terms of ease of management, and architecture.  After all, we’re really trying to do something simple with firewalls: we want bad stuff kept out and good stuff allowed in.  Unfortunately, because there is no easy way to determine what traffic is good and what is bad, we end up going one of two ways.  Either we configure a nice, tight rulebase that’s a pain in the arse to manage anytime something changes, or we don’t make the rule base tight enough, with “permit ip any any” rules all over the place.

So, what’s the solution?  Maybe we can use those new fangled next gen firewalls to make decisions not only on IP and port number but also on the traffic being carried in the TCP or UDP segment?  What this is is essentially integrating IPS into the firewall.  Great idea – it’s a logical step to allow the firewall to make more intelligent decisions.  The trouble is that most IPS implementations are crap.  Even when properly tuned, they make loads of mistakes and generate a massive number of alerts.  In any case, implementing IPS in a firewall chassis normally has a massive impact on firewall throughput – not ideal for Internet edge devices that are already limited in terms of throughput.  Basically what you end up with is a rate-limited firewall with crap IPS capabilities.  No, if you’re going to deploy IPS, do it properly – you’ll thank me in the long run.

How about integrating AV/AS into firewalls?  Well, again, this helps a bit, but it’s still based on signatures and therefore will catch a high number of false positives.  And checking traffic for worms and other malware is just as resouorce intensive as running IPS, if not more.

I think the key thing is to consider what role the perimeter firewall is actually carrying out.  Essentially firewalls dump a load of “known-bad” traffic to stop it getting into the internal network, but they don’t do so well at actually looking inside allowed traffic to see whether or not there’s any risk associated with allowing it through to the DMZ.  Firewall vendors will _claim_ that their products do deep packet inspection, but you’d better hope they’re not doing too much or your firewall is going to be spending too much time analysing allowed traffic and not enough time switching packets.  Fundamentally, most enterprises will have to allow at least HTTP and HTTPS through their perimiter.  HTTPS in particular is a problem, becuase that traffic can’t be inspected unless the HTTPS is intercepted on the firewall – essentially an authorised man in the middle attack.  By accepting that we can’t expect the firewall to do much more than act as a relatively crude packet filter, we can move onto designing the Internet architecture to take advantage of other security tools to protect the environment.  By using a more layered approach to network security, we can also relax a bit more about our firewall rules and make them more manageable.

What we need is a simple metric to decide whether traffic is “good” and should be allowed in, or “bad” and should be dropped.  There’s a lot to be said here for the concept of authenticating traffic, so that even if the traffic source is spoofed, don’t automatically let it in.  We might get to the stage eventually where some sort of heuristic analysis on traffic is realistic in real-time, and that would be a very cool way of deciding whether traffic should be allowed through or not.  Don’t hold your breath though.

I’m not saying that firewalls are totally obsolete (although I do think they won’t be around for all that much longer, at least in their current form).  I think that as the need for NAT disappears with the rollout of IPv6, there will be less of a need for dedicated firewall appliances, which would mean that most firewall tasks could be carried out by the edge router – install a broad security polciy on the edge router and use other security tools inside to filter the traffic properly.  We may see some more multi-role appliances integrated into the Internet perimeter, but to be honest, I’ve never been massively convinced that the UTM (Unified Threat Management) approach was all that good.  Maybe that was a factor of poor integration and execution, but I’d still prefer my WAF to be separate from my IPS, not least because I don’t necessarily want the IPS, WAF, and DBF to be inspecting the same traffic.

And while we’re on this topic, get all this stuff virtualised – I’m sick of rack mounting entire server cabinets full of physical security appliances!

–nta

The other day, I decided to buy an extended life battery for my Sony Vaio.  Since it’s an older laptop, the orignal battery was beginning to show its age and was giving me about half an hour of use before dying.  So, off I went and bought a 13 cell model from what appeared to be a UK site, and much cheaper than batteries Sony sells.  Anyone who has recently tried to buy an after-market battery for a Sony laptop recently probably knows exactly what I’m about to say.

Sure enough, the battery turned up and I plugged it in happily to the back of the laptop, only to see the battery light start flashing like mad.

“Hmm,” I thought “this can’t be good”.  Sure enough, Ubuntu reported that the battery was detected and connected to the mains, but wasn’t actually charging.  This led me to hit Google to see if anyone else had problems.  It turns out that Sony doesn’t like you using 3rd party batteries.  Most vendors would just give you some nonsense about the dangers of 3rd party parts and leave it up to you, but not Sony.  They have actually implemented a hardware block which checks to ensure the installed battery is a genuine Sony part.  Since this is implemented in the Insyde H2O “BIOS” (actually an EFI system), the only way around it is to patch the BIOS.  Now, this sort of thing gives me the heeby jeebies: a 3rd party “patch” for my BIOS?  No thanks!  Still it was either that or send the battery back.  As it turned out, my model wasn’t supported by the BIOS patching tool supplied by the vendor, so the battery is going back anyway.

Even worse, the laptop is now bricked due to a failed BIOS upgrade.  Most Insyde machines have an emergency or “crisis” recovery mode, which allows a reasonably canny user to recover the BIOS to an original image.  Sony, in their wisdom, have disabled this, their only solution – a motherboard replacement.   As a result, I’ve ended up a good portion of my precious time trawling BIOS mod forums and trying to manually edit BIOS files using a variety of suspect tools and hex editors, and after many hours, I’ve finally given up.  Given the age of the laptop, a mainboard replacement isn’t really an economical option, so I’m stuck waiting for a decently priced board coming up on eBay.

I’d love to say that I understand vendors wanting to hide the complexity of advanced BIOS options from their users, but actually I can’t.  It doesn’t aid security in anyway, and it certainly doesn’t help when well-documented recovery mechanisms are disabled so that the vendor can punish their customers with outrageous “repair” fees.  As it is, I’d love to get my hands on the person who came up with the concept of implementing EFI hardware blocks.

It will be a long time before you see me with another Sony laptop, that’s for sure!

–nta

Those of you with a long media memory will remember that back in January this year, IANA ran out of new IPv4 prefixes to assign to the regional registries.  In Europe, that’s RIPE and, fortunately for those of us based in Europe and the US, ARIN and RIPE still have a few prefixes available to hand out.  Organisations trying to get IPv4 space in Asia aren’t so lucky.  APNIC ran out of prefixes to assign to ISPs and other providers in April this year, and RIPE is expected to be the next RIR to run out of assignable IPv4 prefixes.

IPv4 Exhaustion on the Horizon

When RIPE runs out of new address blocks to assign, the IPv4 space in Europe will become very competitive, as ISPs try to hoard their last few IP prefixes for their own use, as well as that of their enterprise customers.  This will likely result in two to three years of IPv4 space being available while Tier 1 providers hand out the last of their space.

It seems that with the exhaustion of IPv4 space, the time has finally come for IPv6.  Although many IPv6-phobes continue to churn out alternatives to a migration to IPv6, there is yet to be any sensible proposal that would be more than a sticking plaster solution to the problem. An example of an alternative to IPv6 deployment is Carrier Grade NAT.

Attempts to Keep IPv4 on Life Support

We’ve actually been here before – back in 1993, the IETF realised that the existing classful Internet addressing scheme wouldn’t be sufficient to maintain the IPv4 address space indefinitely, and RFC1519 came out, allowing IPv4 subnets to be distributed in a classless way.  In 1994, realising that CIDR would not conserve all that many IPv4 addresses, due to pressures such as routing table size, RFC1631 was published allowing the hiding of one IP address behind another.  Eventually, this led to most organisations using RFC1918 private address space internally with a NAT gateway on the outside.  This has meant that most organisations are able to cope with one or two public IP addresses for full Internet access for thousands of clients.  Whilst NAT has undoubtedly kept the IPv4 space alive much longer than would otherwise be possible, it introduced what amounted to an ugly hack into the network, and has been the bane of many network and firewall administrators’ lives.  The fact remains that we now only have a few years to prepare migration plans for much of the Internet infrastructure to IPv6.

However, you might be asking: if there’s years to go before the IPv4 space is exhausted, why should I be thinking about it now?  The answer is that migration is likely to be a complex and unrewarding task.  It is likely that the Internet will essentially be dual-stacked for many years to come – allowing both IPv4 and IPv6 traffic to be carried by the same infrastructure.  The migration of many organisations in Asia and South America (due to a lack of locally available IPv4 prefixes) means that the global migration to IPv6 is already underway.  In order to ensure your users can still connect to the rest of the Internet, you are going to need some way of talking to IPv6 hosts.

IPv6 – A Brief Primer

I’m going to assume that you’ve heard of IPv6 and are aware of its major features, but here’s a few of the critical ones, just in case:

• The IPv6 address space is based on 128 bit addresses for each node in the network.  This means there are 2^128 possible addresses compared to IPv4′s 2^32.
• IPv6 has extensibility built in through the use of extension headers.  This means that it is a much more flexible protocol.
• IPSec has been built into the IPv6 protocol specification and is mandatory on all nodes, meaning that any node can create an encrypted tunnel to any other node on the network.
• IPv6 mobility addressing is built into the specification, allowing a mobile node to retain a single public address whilst moving between multiple provider networks.

IPv6 is a completely new protocol, and although it still carries TCP and UDP traffic like IPv4, is totally different to the older protocol.  For whatever reason, IPv6 is not backwards compatible with IPv4.  That means in the intervening years between the current setup and a fully IPv6 Internet, it’s likely that major portions of the Internet – and your corporate infrastructure – will need to be dual-stacked.  That is: nodes will need to run both IPv4 and IPv6 at the same time.  As infrastructure migrates to the new protocol, dual stack nodes will gradually move to gateway nodes rather than existing on corporate networks.

If you want to play around with IPv6, I can thoroughly recommend Hurricane Electric as both an IPv6 resource and a tunnel broker that allows you to connect a gateway host or router to the IPv6 Internet so that you can use IPv6 internally, even if your ISP doesn’t yet support native IPv6.

What This Means for Your Network

Hopefully by now, you’ll be aware that a migration to IPv6 is inevitable at some point.  As increasing portions of the Internet migrate to IPv6, IPv4 hosts will become increasingly isolated.  It doesn’t necessarily mean that you’ll need to move all your hosts onto IPv6 straight away, or that you need to rush out and buy new network kit (in fact, most modern network devices and operating systems already support IPv6), but it does mean you need to start planning your migration and perhaps running some tests.  Start evaluating your Internet perimeter and especially what benefits IPv6 could bring to your Internet architecture.

Also think about what needs to change about your security architecture – it will be necessary to allow certain types of ICMPv6 through your firewall as they are required for IPv6 to work properly.  Also bear in mind that NAT will be no more.  Although NAT is not a security technology, it has come to be seen as one, and therefore the elimination of NAT will need to be comprehensively understood and any vulnerabilities that this exposes mitigated by your security personnel.

Conclusion – Start Preparing Now!

IPv6 is definitely on the way and preparation will be key to successful migrations.  IPv6 day this year proved that the migration won’t necessarily be as painful as some like to make out, but it’s still going to be a lot of work.  The sooner you start preparing, the smoother things will go when the time comes!

In the meantime, test your IPv6 readiness here

–nta