How to Move a Datacenter in Two Weeks with NetBox

4 minute read

Please note that all players have been anonymized to protect the guilty. Please also note that I have jumped ship several times during my formal IT education. You might see why.

I had just started my apprenticeship at this company—hardly past the three-month mark—when BigBoss told me I was to assist with the move of the company’s servers. About nine racks’ worth of equipment across the city, minimum downtime… you know the drill. I don’t at this point, but I think figuring out what equipment goes where and how it is connected is a good first step.

My direct superior helpfully points me to a folder on our network labeled “Server Documentation”. I see the file ending. These are Excel files. Uh oh.

A quick check reveals that not even half of our switches are in here. Somebody has attempted to visualise the whole deal by drawing rack outlines with the columns. It’s colourful and pretty, yet not a single cable is part of the documentation. It is a complete and utter mess.

I get back to my superior and he agrees that the situation is hardly ideal. I’m just getting known in the company for my research skills, so I’m tasked to find us software that will allow proper documentation of all racks and interfaces.

An hour later, I have identified the two best candidates. First I present my favourite: NetBox, which I knew from an earlier internship, has all the features we need, is self-hosted, and it’s open source!

“Open source?!” BigBoss laughs at that, “Open source software isn’t suitable for enterprise applications!”

“Okay, then we probably need to go with $EnterpriseSoftware, which is hard to administer and doesn’t have all the features we need, but at least it’s way over budget.”

He looks at the costs.
He looks at the capabilities.
He looks at the costs again.

“Do a presentation for the C-Suite. If you can convince them, I’m fine with it.”

Sure, apprentice, new software, C-Suite, software options that cost more than my salary in a year… no pressure!

In the end, the C-Suite wasn’t hard to convince; also, I love convincing people. They liked that NetBox is an off-the-shelf solution for our exact problem and didn’t seem too bothered by the fact that it was open source.

I would have loved to introduce them to the wonders of automation, in the form of the NetBox API, that would have done a lot of the heavy lifting in documenting the datacenter. But this is an old school kind of company, so the very Idea of automating something you could make apprentices do was frowned upon. Not like we had an approaching deadline or anything.

Next came a roll out in our testing environment, a couple “oohs” and “aahs” from my superiors, and a couple “I told you so”s I kept quietly in my head. They loved the multi-tenant option, allowing us to not only document our own racks, but also the customers’ racks, without any possibility of a mix-up.

I think the documentation is solid enough to be able to direct a helper on location (even if that helper is non-technical staff), enabling shorter turn-arounds for any disruptions occurring at remote sites, in turn saving money, time, and sanity of the employee who doesn’t have to drive out to some village.

Then we had to check every single connection by tugging a bit on the cables, taking care not to disconnect the productive environment, screaming over the racket of racks, and slowly untangling the organically grown infrastructure our predecessors had left us with. All told, we spend about 130 manhours just documenting the racks. Additionally, I spent twenty hours with a label printer, putting the UIDs NetBox had generated on the corresponding cables.

Being able to distinguish the kind, colour, length, transmission rate, and interfaces of every cable inside NetBox saved my life, or at the very least our deadline. If a future version could automate sitting on a server room floor and attaching tiny labels to hundreds of cables, I might just marry that version.

Armed with this shiny new documentation—our first single-point-of-truth—we even managed to plan ahead where to put equipment that was yet to be purchased. If the cabling was any indication, planning ahead was a first.

The fact that NetBox allows us to label devices throughout their lifecycle from “staging” to “decommissioning” brought another round of impressed “oohs” and “aahs” from my superior.

For the outsourced moving company, I printed A3 size posters with front and rear views of the racks they had to assemble. They also got binders with plans where each device had to go. I brought enough for everybody, and a couple more for us, documenting the cabling. NetBox even allows exporting everything to CSV files, which was amazing for setting up these binders.

Otherwise the move went almost flawlessly, and the included stress testing of NetBox was so convincing it’s now part of the portfolio at that company.

Seems like NetBox is indeed suited for enterprise applications.

In summary: NetBox did everything we wanted it to, passed our security audit, and has the potential to do so much more, with options to document IPs, VMs, and entire networks down to the providers.

By the end, my superior acknowledged that the move was pretty much my project more than his, so if a tool can do that for an apprentice three months in, imagine what it can do for you.


Announcing the NetBox Advocates Program

1 minute read

Are you a NetBox power user? Maybe you’ve even built a plugin or custom script? Would you like to be more involved in feature development and roadmap planning for NetBox? To better enable community members like you to have a direct impact on NetBox’s development, today we’re launching the NetBox Advocates Program!

What does it mean to be a NetBox advocate?

  • We’ll occasionally reach out (via email) for feedback on specific new or planned NetBox features. For example, we may ask for feedback on the plugins development experience, or which of several potential new features you’d prefer to see first. This feedback will help guide NetBox’s long-term development.

  • You may be offered access to pre-release instances of NetBox or other tools for evaluation and feedback.

  • You’ll be able to leverage our community Sentry service for error reporting and correlation. We’ll proactively reach out to advocates who opt to enable this service regarding any new errors we detect. We may ask if you can assist with reproducing the error to help develop a fix, or point you to a potential fix.

  • Your organization’s logo will optionally be published in our list of advocates.

And this is just the beginning! We fully expect the importance of the advocates program to grow as the NetBox ecosystem continues to expand.

Register here to get started!


Error Logging with Sentry

1 minute read

One of the challenges that comes with developing open source software is getting reliable and detailed bug reports from users in the wild. Because NetBox is generally self-hosted and has no phone-home function, we don’t have any direct access to deployments other than what we (the maintainers) deploy ourselves. But there’s a new feature in NetBox v3.2.3 that I think can really help us out: Sentry intgeration.

Enabling error reporting via Sentry allows us to collect and analyze exceptions and other errors as they happen, rather than having to wait for bug reports submitted by end users. It won’t capture all bugs, of course, but it will alert us to the more severe issues. Sentry reports include a full stack trace, which is often needed to determine just where a bug exists, but no end user-identifiable information or confidential data.

To enable Sentry integration, simply turn on the SENTRY_ENABLED configuration parameter and restart NetBox:

SENTRY_ENABLED = True

NetBox will begin generating error reports and sending them to our centralized ingestor. One of the biggest benefits of this community-powered arrangement is that we’ll be able to correlate bugs by software release, operating system, and other deployment attributes.

An example Sentry error report

Of course, it’s possible to use your own Sentry ingestor too: You’ll just need to define your custom DSN:

SENTRY_ENABLED = True
SENTRY_DSN = "https://examplePublicKey@o0.ingest.sentry.io/0"

You can find more detail on Sentry configuration in the NetBox documentation.


NetBox v3.2.3 Released

1 minute read

NetBox v3.2.3 is now available on GitHub!

Enhancements

  • #8805 - Add “mixed” option for device airflow indication
  • #8894 - Include full names when listing users
  • #8998 - Enable filtering racks & reservations by site group
  • #9122 - Introduce clearcache management command & clear cache during upgrade
  • #9221 - Add definition list support for Markdown
  • #9260 - Apply user preferences to tables under object detail views
  • #9278 - Linkify device types count under manufacturers list
  • #9280 - Allow adopting existing components when installing a module
  • #9314 - Add device and VM filters for FHRP group assignments
  • #9340 - Introduce support for error reporting via Sentry
  • #9343 - Add Ubiquiti SmartPower power outlet type

Bug Fixes

  • #9190 - Prevent exception when attempting to instantiate module components which already exist on the parent device
  • #9267 - Remove invalid entry in IP address role choices
  • #9296 - Improve Markdown link sanitization
  • #9306 - Include VC master interfaces when selecting a LAG/bridge for a VC member interface
  • #9311 - Permit creating contact assignment without a priority via the REST API
  • #9313 - Remove HTML code from CSV output of many-to-many relationships
  • #9330 - Add missing module_type field to REST API serializers for modular device component templates

NetBox v3.2.2 Released

1 minute read

NetBox v3.2.2 is now available on GitHub!

Enhancements

  • #9060 - Add device type filters for device bays, module bays, and inventory items
  • #9152 - Annotate related object type under custom field view
  • #9192 - Add Ubiquiti SmartPower connector type
  • #9214 - Linkify cluster counts in cluster type & group tables

Bug Fixes

  • #4264 - Treat 0th IP as unusable for IPv6 prefixes (excluding /127s)
  • #8941 - Fix dynamic dropdown behavior when browser is zoomed
  • #8959 - Prevent exception when refreshing scripts list (avoid race condition)
  • #9132 - Limit location options by selected site when creating a wireless link
  • #9133 - Upgrade script should require Python 3.8 or later
  • #9138 - Avoid inadvertent form submission when utilizing quick search field on object lists
  • #9151 - Child prefix counts not annotated on aggregates list under RIR view
  • #9156 - Fix loading UserConfig data from fixtures
  • #9158 - Do not list tags field for CSV forms which do not support tag assignment
  • #9194 - Support position assignment when add module bays to multiple devices
  • #9206 - Show header for comments field under module & module type creation views
  • #9222 - Fix circuit ID display under cable view
  • #9227 - Fix related object assignment when recording change record for interfaces