MakerBot Server and Network Clean-up
Makerbot is a global leader in desktop 3D printing. But when its office wifi went out during a holiday party in 2018, the VP of Operations, Jim Franz, found himself digging through a “spaghetti mess” of cabling in the server room trying to reset the wifi controller.
The room was obviously an eyesore. Franz says when he started in June 2018 he knew a cleanup was in order. Senior IT Support Engineer Brian Aronson says “it was scary in there” — ports in the patch panels weren’t connected to the network and doing any troubleshooting involved “a lot of guesswork just to know what VLANs were on.”
“If I’m in the server room I am probably freaking out about something,” says Aronson. “When you’re under pressure you don’t want to have to face a hostile environment.”
Macktez was brought in at the start of 2019 and found more wrong than just appearances. There were constant network issues blamed on the internet service provider that were more likely due to internal misconfigurations. Phone calls dropped. Downloads were too slow. Network and VLAN assignments were not documented. 10G interfaces were not being utilized. And the system had no redundancies, so if any one part broke the whole network could go down.
“The key,” says Macktez CTO Reilly Scull, “was that the equipment was all top-tier. But there was no unified network architecture that emphasized performance.”
“All the parts were in place,” says Reilly, “and the will to fix the problem was readily apparent at MakerBot.”
The first 20% of the project was spent evaluating different options. Reilly spoke with manufacturers to determine optimal server configuration. Consultant Scott Battaglia also addressed switching capabilities to make sure that existing equipment was optimized.
Reilly determined that the entire server and network racks would need to be dismantled and reassembled, resulting in a week of downtime. MakerBot leadership agreed that the only time that could happen was the last week of December, when the company was on holiday.
During that week, Reilly worked closely with Consultant Lauren Steinmeyer for 12-hour days to finish the job in time for MakerBot’s return after New Year’s. They upgraded dozens of servers, rebuilt VMs, reconfigured storage arrays, and implemented a brand new backup strategy.
And yes, they also cleaned up the wiring. They moved all power management to the back, connected network equipment with shorter patch cables, and ensconced longer cables in braided sleeves and secured them to the rack where appropriate with zip ties and velcro straps.
“It looks like the control room of the Death Star!” That’s the text Reilly got from Aronson when he saw the new server room. (We’re pretty sure Aronson meant this as a compliment.)
Aronson says, “The immediate thing I noticed was I was a hell of a lot less anxious in there.”
Besides the aesthetics, the infrastructural benefits are profound: 10G interfaces are now fully utilized for lightning-quick internal transfer speeds; throughput has been increased 400%; VLAN assignments are now consistent and secure.
And just as important for MakerBot, its IT team is now empowered to fully utilize its own equipment. “This past year has been about rebuilding confidence in the IT department,” says Aronson. “We have a lot more power in there, more capacity — future projects and growth, that’s where it’s going to be a big benefit.”
Says Franz, “We are now set up for the future (and no longer ashamed to show people the server room).”