1,095,038,164 Package

The story of a billion downloads

The open-source journey of Spatie

Currently, we have around 300 open-source repositories on GitHub. Our packages have been downloaded a staggering 1,095,038,164 times. That's almost 45 million downloads each month!

Putting our modesty aside for a minute, this is quite an achievement for a small team; we're only 10 people at the moment.

This all did not happen overnight. Read on to learn the story behind our open-source efforts.

Chapters

  1. Hello Laravel
  2. Why packages?
  3. Maintaining open-source
  4. Why use Spatie packages
  5. Favorite packages of our team

Hello Laravel

Up until 2013, all Spatie projects were made using a custom-built CMS that was powered by Zend Framework 1. In 2013, we transitioned to Laravel, which was at version 4.0 at the time. We were immediately impressed by the expressive syntax and the big focus on developer happiness of Laravel.

At that time, a lesser-known guy named Jeffrey Way started his next project, Laracasts: a video tutorial site dedicated to Laravel. We realized that his videos could greatly accelerate our learning process and immediately bought a lifetime subscription.

On March 4, 2014, Jeffrey published a video titled "Continuous Integration With Travis." Travis and its integration with GitHub looked so cool that we wanted to use it.

A few months before he made the Travis video, Jeffrey created a mini-series on package development that contained all the information you need to get started creating your own package.

Making the switch

At the time we were working on a project that needed to automatically create screenshots of a website. With the things we learned on Laracasts, we started to work on our first package: Browsershot. This package was essentially just a simple wrapper around PhantomJS (the current version uses Headless Chrome and Puppeteer).

It was pretty exciting working in the open. We were thrilled every time the download counter got up and were incredibly happy that other people started writing about it.

Did you know?

Our first package, browsershot, is currently on its 4th version and has been downloaded more than 13 million times since its release!

Back when we used Zend Framework, we used a custom-built application template, called Blender, to kickstart all our client projects. Think of it as a very basic CMS. Besides traditional CMS features, Blender could do a lot of cool things: pull in data from Analytics to show graphs about the usage of the site; handle uploaded files; subscribe users to a MailChimp list and more.

To power new projects, we needed a Laravel version of our CMS and started creating it from scratch.

We soon realized that some of the things we were porting to Laravel could also be useful to other developers and decided to build these functionalities in packages.

In quick succession packages like laravel-newsletter, laravel-medialibrary and laravel-analytics were created, each of them still being supported and regularly updated.

Currently, every new package that we create gets born inside a client project. In almost every project we find some functionality that can be extracted to its own package.

Why packages?

There are many benefits of creating packages that make time working on open-source code well spent.

A lot can be learned while creating a package. Each package needs to be carefully crafted. It provides clear and understandable syntax. Like Laravel, we want all our packages to put a great emphasis on developer happiness. They should be easy to use. They should have great documentation. They should have clear tests. Thinking about how other people will use your code will make you a better developer.

Improving our work

The issues reported and the PRs submitted by the users of our packages provided another chance for learning. People can point to mistakes that are in our packages and can propose interesting new features that we didn't think about.

Of course, we're also dogfooding ourselves. Our packages get used on most projects. If we discover a bug in a package used in a project we can very quickly fix that and distribute that fix through the power of Composer to our other projects.

There are also commercial benefits. In recent years, we've landed some cool projects thanks to our open-source work. This shifted our focus to working for international companies, including some well-known ones. These new clients often already use our products and packages.

While our main aim with open-source efforts isn't to attract clients, it's certainly a pleasant bonus.

Did you know?

The number of downloads of our packages is constantly growing. We reached 10 million in 2017 and 100 million by 2021.

Maintaining open-source

When we meet people at conferences, we often get asked how it is possible for a small company like us, only 10 people big, to create and maintain so many open-source packages.

The simple truth is that it does take up a lot of time. Creating the code for the package itself, writing tests, writing documentation and publicizing distribution all require quite a bit of attention.

When the first stable release of a package gets tagged the work is not over. Maintaining a package, responding to issues and reviewing PRs takes a lot of time (and dedication) too.

Did you know?

Over the years, we responded to more than 12,000 issues and reviewed 18,000 PRs.

Most of our packages are created while working on client projects. If we see something that can be made into a package, we do it. Because we've done it so many times, we're pretty quick at it, too.

Making time

A big way for us to keep our open-source work manageable is the way plan our week. Each week, we plan for only four days ahead, leaving one day flexible. This day isn't fixed; its time is spread out across the week.

On this fifth day, everyone in our company can work on open-source projects or their own tasks. Though one day might not seem like much, when done consistently each week for many years, it adds up to a significant amount of time. We've been following this approach for open-source work for around a decade now.

Experience and accountability

Every package has a primary author, the person who initially wrote it.

Although packages are reviewed by other team members and everyone helps each other, it is the primary author who does most of the maintenance work. At Spatie, we are fortunate that staff turnover is very low and most of the people who wrote a package are still with our company.

When we create a package, we always make sure we have a test package for it as well. That way we know that our code is stable. We also know that good documentation is essential, so we dedicate time to this as well.

Because we provide testing and good documentation, the first release of a package is often already very stable and does not need much maintenance after that.

Did you know?

Our most popular package that we never use ourselves is laravel-permissions, a great way to manage user rights and roles in a database. :-)

Keeping it maintainable

Even after a package is released, we aim to keep the scope small. The key to keeping everything maintainable is to say no to PRs by default. We accept PRs that are either very simple or add features that we see ourselves using.

We enable Discussions on Github for most of our packages. This is a great way for people to ask questions and request features. We also often engage in these. The issue tracker is used to track work that we would like to do on a package. We try to keep that issue list small.

In summary, the big secret to creating and maintaining so many packages is consistency. By having everyone in our company working on open-source regularly, we get a lot of work done.

Why use Spatie packages

Adding a package to your composer.json should not be done lightly. You need to be sure that a package works correctly, can be used easily and that it will be maintained well over time.

Every Spatie package adheres to a list of rules we have set for ourselves, ensuring that when you use a Spatie package, you always have a great experience.

  1. We have put a lot of effort into making our package as user-friendly as possible. Every method should have a function. Ultimately, a package should be fun to use.
  2. We invest time in creating excellent documentation for each of our packages. This makes it easy to discover what a package does and what all the features are.
  3. All our packages have an extensive test suite. This provides you (and us) with proof that the package code behaves correctly and works under all conditions.
  4. In our opinion, code readability is very important. We structure and write our code in such a way that others can easily understand what is going on. Names of variables, functions and classes are chosen with care. You should be able to understand how the package works just by reading the code.
  5. We know we can’t imagine every scenario our code will be used in. That’s why we make our packages as customizable and extensible as possible. We don’t use the final keyword, and make everything protected by default instead of private. Our code is structured in small functions and classes that can be overwritten or extended. We see our users as responsible programmers who can be trusted with sharp knives.
  6. When creating a package we intentionally keep the scope small. In most cases, packages are created to be consumed in our projects. We would rather have a small package with one very polished feature, than a large package that tries to cover all possible edge cases and variations of a problem.
  7. Most of our packages are maintained for a very long time. When a new PHP or Laravel version comes out, our team spends time to make sure the test suite is running correctly and that a new package version is tagged. Usually, all of our Laravel packages are compatible with the major releases of the Laravel framework within a few days.

Favorite packages of our team

We asked our team members a simple question: what is your favorite Spatie package that you have used or worked on, and why?

At Spatie, we are a fan of Inertia. This package truly shines when we need to define types for both PHP and TypeScript, and keep them in sync. By parsing PHP files and transforming them into TypeScript type declarations, we save time and mental effort but also streamline the refactoring process.

It was the first Spatie package I've ever added to one of my composer.json files. At that time, I wasn't working for Spatie yet, but I loved the idea of such a simple package. It has only one use case: take the field of an Eloquent model and make a slug from it so that the model can be queried with that slug. The docs are excellent, and the API is easy to use. It is the perfect example of a terrific Spatie package. And it, in the end, made me start working at Spatie.

For saving me from the dozen times I wondered 'Why is my change not reflecting?'' when I first started to work with queues.

Despite being abandoned, the Blade X package now lives on in Laravel. It added custom HTML components to Laravel before Blade components were natively supported. Hacked together on a Friday afternoon when both project managers were MIA, this package started with an friendly discussion over lunch and ended with the entire Spatie team ditching 'real work' (who needs that, right?) to tinker with this gem. I love how an impulsive "there's nothing stopping us from doing this weird thing"-idea ended up with 180.000 downloads thanks to a fun afternoon with the team.

I love playing with off-beat ways to make tests more readable and joyful to write. Snapshot testing was popularized by Jest to test UI components. I thought it would be a great fit for testing large data, so I wrote a PHPUnit package to bring it to PHP.

The name says it all, it makes exporting and importing Excel & CSV files simple. Every project needs some kind of importing or exporting, this package just makes it a breeze.

At Spatie, we use Media Library in every single project. I still remember I was still learning some basics of Laravel when coding up the first version of the media library. For the media conversions, queues are used, and this was the very first time I used queuing which felt very empowering. Currently, the package is already at v11, which is a nice indication of how much we interacted on this package, it's well polished now.