Nexus to Artifactory

Nexus to Artifactory – An Infrastructural Digest

Nexus has been the heart of Camunda’s artifact storage for more than a decade. Artifact storage is a service used by our engineering teams to publish their software and distribute it internally and externally, which includes both enterprise (private) and open-source (public) software.

Looking Backward

Let’s take a look at the past:

  • 2010: The first Nexus artifact storage was set up by our co-founder Bernd Rücker and started off as a WAR deployment.
  • 2012: Disks were lacking enough capacity to hold all the artifacts.
  • 2013: The instance was made accessible to our enterprise customers.
  • 2015: Due to a lack of a WAR supply by Sonatype, the deployment was switched to a Nexus built-in Jetty version.
  • 2015: Another disk issue occurred due to lack of space.
  • 2016: Migrated to its own server with an expected growth of 250 GB/year. Plot twist, it didn’t last.
  • 2018: Ran out of disk space again.
  • 2018: Deployment was switched to Docker to incorporate easier upgrades and additional isolation.
  • 2019: Upgrade from Nexus 2 to Nexus 3.
  • 2019: Switch to Solid State Drives (SSD) to be able to handle more concurrent requests, especially during the daily backups as this IO heavy task made the Nexus a lot less responsive.

Most of those issues resulted in growing a virtual machine and moving everything to another disk or server. This caused us to switch the underlying infrastructure at least six times. At the end of the day, it was always followed by a lot of engineering hours and cross-team collaboration to plan and migrate to a new infrastructure. In this case, a special shout-out goes to our IT team for handling those matters quickly and sometimes on the weekends to keep the user impact low.

The upgrade to Nexus 3 was done two years after the first major release and five years after its first preview release. For you to understand why it took this long, one must know that the responsibility of Nexus has been transferred a couple of times over a decade, following a couple of restructurings with it finally ending up with the newly formed Infrastructure team in 2019. Overall, more than 13 different engineers have worked on Nexus within a decade, of which some have left Camunda and others hold completely different positions here now.

Looking Forward

At the beginning of 2021, we started looking into the possibility of either bringing Nexus to the cloud or possibly switching to a Software as a Service (SaaS) product as long as it satisfies all of our requirements. In March of the same year, we decided to go for the SaaS enterprise product of JFrog Artifactory. This decision led to this blog post and our announcement that we’re moving from Nexus.

JFrog Artifactory having a SaaS option was a major factor in the decision, which means less engineering overhead for us. We also value the support it offers for Configuration as Code (CasC) by either utilizing Terraform or a YAML endpoint. This works better with our current approach of handling infrastructure compared to everything previously being done manually. Certainly, there are more details to share, but to keep it short, we simply saw JFrog Artifactory as a better fit for our current needs.

For more than a decade, Nexus has been hosted under the domain of https://app.camunda.com/nexus/, which we aim to continue by proxying the requests to new routes in JFrog Artifactory.

What’s In It for the Users?

In the long run, we expect more consistent availability for our users due to zero downtime upgrades and scalability. Additionally, by using JFrog Artifactory in the SaaS option, we have growth potential by replicating the data to other regions or utilizing edge nodes in the future. This would directly impact the accessibility and speed of the provided artifacts.

What’s Next?

The migration will happen Oct. 15, as we have to migrate over a terabyte of artifacts to the new infrastructure, which will hopefully be the last time that we have to do this for the next decade.

From Oct. 15 – 17, Nexus will be put in read-only mode, meaning everything will still be accessible as usual.

For users, this means that from Oct. 17 on, you’ll be served by the JFrog Artifactory instance, which will be a smooth transition as nothing is required on your behalf. You will still be able to use Lightweight Directory Access Protocol (LDAP) for authentication and authorization to access the enterprise software published by our engineering teams.

Some additional information regarding accessing JFrog Artifactory via Maven can be found in our documentation.