LinkedIn open-sources Ambry, an object store for media files

Ambry.


LinkedIn today announced that it has open-sourced Ambry, a piece of software it built to store and serve up media files like photos, videos, and PDFs. The system is available on GitHub under an open-source Apache license.

 

LinkedIn previously relied on a complex architecture involving closed-source technology that was not cheap to scale, even as user numbers and data have both kept increasing. It wasn’t easy to expand, either.

 

“This convinced us that we needed to find a better approach,” LinkedIn engineering manager Sriram Subramanian wrote in a blog post. “We took it as an opportunity to look at different end-to-end solutions. We looked at many possible solutions including distributed file systems, storage appliances, cloud, and an in-house implementation. We thought deeply about our design goals, understood the tradeoffs and were convinced to build an in-house solution that fit our needs better.”

 

The system has been in production for more than a year, wrote Subramanian, who has previously worked on Kafka and Samza open-source software during his time at LinkedIn.

 

Of course, other big technology companies, like Facebook and Google, regularly open-source different types of software. But object storage isn’t the biggest area of innovation in open-source world.

 

Many popular Web services, like Airbnb, Pinterest, and Spotify, look to cloud services like Amazon Web Services’ (AWS) S3 for object storage. It’s not trivial. But like LinkedIn is more like Facebook and Google in the sense that it operates its own infrastructure for its services. Hence the need for full software that can operate across multiple LinkedIn data centers. Of course, there are representational state transfer (REST) application programming interfaces (APIs) for working with Ambry, which is written completely in Java.

 

The system is complex. For a full rundown of its functionality, check out the full blog post and the Ambry wiki.

 

 
 

Get more stories like this on TwitterFacebook

 
 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s