Updating Multiple Systems
In a previous role as Linux Admin for a mom-and-pop business I managed several Linux systems. On the server side were some Dell rack servers running Debian based Proxmox with LXC containers and KVM virtual machines (VMs). On the user side were workstations and laptops.
All servers were CentOS and Debian with some Windows VMs. The workstations and laptops were Debian.
Much of the role was remote from home. To be consistent with the business environment and “eat the dog food” I used a Debian VM.
All of the systems were “pets” rather than “cattle.”
Systems were updated in a methodical roll out manner. Some test systems were updated first. If that went well then next came some selected low impact production systems.
There were multiple Proxmox hosts, backup servers, name servers, and DNS caching servers. Systems of the same function were not updated on the same day. That way if anything went awry the other similar systems were not yet affected by the updates. Typically the Proxmox servers were updated only two per day and only one per day with the backup servers.
The workstations and laptops were configured to auto update only in the evening after business hours. That way nothing interrupted the users during the day. I never got bit by that strategy although there was the infamous GRUB boothole patches that were hastily pushed without full testing. Fortunately mail lists and RSS feed subscriptions forewarned the patch fiasco ahead of time.
The laptops were configured to auto update only when connected to the office subnet. That avoided interruptions in the field. For updates the technicians connected the laptops to the office subnet every other week.
All systems, including workstations were backed up daily and laptops whenever connected to the office subnet.
Custom shell scripts helped create new systems. For the workstations and laptops there was an imaging system. In a worst case scenario those systems could be rebuilt and restored in less than hour.
Posted: Category: Usability Tagged: General
Next: lddlibc4 command not found
Previous: New Network Switch