How we streamlined Code Generation at Lema
Hello fellow developers,
Today, I want to share an exciting journey our team at Lema has undertaken in optimizing our workflow. We use two powerful tools, ent and buf, for modeling, querying data, and managing Protocol Buffers (Protobuf) schemas. In this blog post, I'll walk you through our experience of transitioning from keeping generated code out of version control to embracing its inclusion in our Git repository.
The Genesis: An Overview of Our Tools
Before we dive into the heart of the matter, let me introduce you to our key players – `ent` and `buf`. `ent` is a robust Go ORM (Object-Relational Mapping) tool that aids us in modeling and querying data, while `buf` is a Protobuf toolchain that generates code, ensures backward compatibility, and enforces linting for Protobuf schemas.
The Transition: Pros and Cons
Pros
1. Seamless Branch Switching
In our initial setup, changing branches necessitated a manual call to make generate. Now, with our generated code committed to Git, this step is no longer necessary. It's a small change but makes branch switching a breeze.
2. Simplified CI/CD Workflows
- Reproducible Builds: Storing generated code in Git ensures reproducible builds. There's no chance for discrepancies in the generated toolchain version.
- No More Makefile: We started out with a Makefile for all operations on our code base but after we detected that it is a magnet for utility scripts, we decided to get rid of it. It got to a point that the only function left in the Makefil was our code generation scripts. With the generated code in the repository, the Makefile is not needed anymore, and code generation is seamlessly integrated into our CI/CD pipeline.
3. Effortless Deployment
Without the need to call make generate before building, our deployment process has become more straightforward. This improvement simplifies our deployment pipeline, eliminating the last hurdle for a smoother release process.
4. Validation with Confidence
We've embraced a practice recommended by ent – verifying that all generated files are checked into Git. This ensures that everyone on the team is working with the same codebase, reducing the chances of discrepancies.
Cons
1. Git Repository Size Concerns
The decision to commit generated files to Git raises concerns about repository size. While it can contribute to bloating, the trade-off for streamlined workflows and consistent builds is well worth it.
2. Protoc Version Maintenance
To avoid any mismatch issues between users, it's essential to manage and maintain the protoc version across the team. This ensures that everyone is on the same page when it comes to Protobuf schema compilation. Not doing that can cause extra git conflicts
Conclusion
Optimizing workflows is a continuous process, our journey from excluding generated code from Git to embracing its inclusion has significantly improved our development and deployment processes. While there are considerations such as repository size and protoc version maintenance, the overall benefits have proven invaluable.
If you're contemplating a similar transition in your project, weigh the pros and cons, consider your team's specific needs, and, most importantly, keep coding efficiently!
