Cutting Costs by "Semi-Outsourcing"

The place im working at is into some serious cost-cutting (what business isnt). Recently the people way above my head had a meeting with some account managers from Sun. I can imagine the meeting going something like this:

Sales rep: We can help you achieve better ROI on spent $$$$'s by letting us do more for you but you will still pay the same amount for your Service. Let us help you keeping your Solaris more available and more secure by handling all your OS updates.

Manager: That sounds like a good idea, Id like more $$$$'s to spend on our yearly Golf-tournament.

Sales rep: Thats what we thought, do you want some complimentary golf-balls?

So the meeting trickles down to us the senior admins and we start to groan. My opinion is this:

Keeping 500 unix boxes up-to-date with the latest patches is NO problem whatsoever, aslong as you dont have to think about applications. Ive seen several different patching tools that logs into machines, checks all patches and installs newer ones if needed and then makes a reboot and the machines is back online in no-time. (JASS, N1 *cough*, flar's etc etc). Sounds great on paper and when Sales Rep's talks to Management people. But in the end it comes down to the following 2 problems:

Applications and Service-windows.

Ofcourse all you other people out there work for THE company that has full control over their SLA's and all have their little service-windows every month when its ok to take down the servers and apply OS and security patches, and all your system owners scream of joy when you say you want to apply your quarterly OS patches.

Why cant management understand that Solaris is easy, applications are hard, you cant apply patches and go home and expect all applications to work as intended when you come back in the morning. After applying patches you need to make sure that the application works as intended. What takes time when it comes to OS/Security patches the troble is getting access to the system, getting some allowed downtime and then making sure everything works as intended after the patches have been applied, GETTING THE PATCHES ON THE SYSTEM IS EASY.

What makes management think that Sun's technicians would do a cheaper/faster/better job at patching our systems? When the problem lies within our own organisation? Hire us some secretaries to keep the paperwork away from us and that can handle booking of downtime and test-personell (or help with some automatic testing tools), and we will make sure that patching goes smoothly from thereon and everafter!!

Solaris Zones, just how seperated are they anyways?

So I'm looking into BSM Auditing on Solaris 9 and 10. Anyone noticed that there is something missing in Sun's great offering? Well yes, a nice way of collecting the audit logs would have been nice.

Some might say, Solaris 10 supports syslog! Which is a great improvement, until you realise that syslog truncates each entry at 1024 characters (which isnt THAT long if you have looked at the audit logs).

So what to do if you want to collect logs from several hundred of servers in various security zones? Sun suggested that with Solaris 10 you can run your services in a Zone and the auditing will then take place in the global zone and then you can make a few scripts to send your logs to a logserver using scp, thats a pretty good approach but at the place where I work we dont really like having to open for ssh to a logserver.. What other ideas can we come up with?

One could be, Solaris 10 on a host with a few network interfaces and a zone for each interface connected to each DMZ running a NFS-server, each server logs to the nfs-share and rotates the log every 5 mins. On the global Zone a cronscript checks with fuser if each file is being held by the auditd and if not decides to move it of the NFS-share into the global zone, the files cannot be deleted without breaking into the global zone of the logserver which shouldnt be accessible from the different DMZ's. Sounds pretty neat in my ears and quite cost effective without having to resort to buying a StorEdge 5310 with Compliance Archiving software for quite a couple of thousand dollars..

So Im quite happy wow what a nice idea.. I make a quick phonecall to a my Sun Contact and explains what Im thinking and he responds, Nice idea.... On paper..... Only problem is that you cant have seperate NFS-Servers running in different zones.. and then my mood just went south from there.. which brings me back to my headline..

How seperate are Zones anyways? How dependent are they upon eachother, they share kernel, for some reason they arent seperate enough to run nfs-servers in them, you cant have different system clocks in them, they share Shared Memory.. How much can we trust zones, if Sun themselves cant make their own protocols and services run within a Zone (they released the NFS-specs to the opensource world in -84 apparently), how likely is it that all 3rd Party vendors will succeed in writing Zone compliant software?

Dont get me wrong, I think Zones is a great leap forward but is it "separate" enough, I dont have enough competence to judge if it is or isnt, but from my point of view, perhaps they arent in the current form, maybe they will be in Solaris 11?