Boost CI Speed & Quality: Unit Tests & Advisory Gates

by Admin 54 views
Boost CI Speed & Quality: Unit Tests & Advisory Gates

Hey everyone! Let's dive into how we can turbocharge our Continuous Integration (CI) pipeline and make our lives easier when working on the project. We're talking about a few key improvements: adding a super-speedy unit test check that's required for every pull request (PR) and relaxing some of the less critical checks to advisory status. This should give us quicker feedback on our code changes and reduce some of the noise in our PR process. Let's break it down, shall we?

The Need for Speed: Fast Unit Tests

First things first, why are we even doing this? Well, nobody likes waiting around for ages to find out if their code is good to go. Currently, our CI runs a full suite of checks, which takes a bit of time. This includes things like security scans, code analysis, and of course, unit tests. While all these checks are super important for ensuring code quality and security, the longer the CI takes, the slower the development cycle becomes. That's not ideal, right?

To address this, we're introducing a fast unit-only job. This new job will run the essential unit tests for the jive-api and jive-core components. The key here is speed. Because these tests are focused solely on unit tests, they should run super quickly – ideally in under 3 minutes. This will give developers immediate feedback on whether their code passes the most fundamental checks. If the unit tests pass, we know the code has a good chance of working, and we can proceed with other reviews. If the unit tests fail, the PR is blocked, preventing further checks until the issue is resolved. This helps prevent larger issues down the line.

The idea is simple: get quick feedback upfront. If the unit tests pass, we're in good shape and can move forward with other checks. If the unit tests fail, then we're immediately notified, allowing us to fix the code without wasting time waiting for the longer checks to run. This new unit test job is going to be required. Meaning, a PR can't be merged unless the job passes. The unit tests ensure that the most important checks pass before the merge. This is a game-changer for speeding up the development process.

Implementation Details

This new CI job will run the following tests:

  • jive-api unit tests: SQLX_OFFLINE=true cargo test --lib
  • jive-core unit tests.

We'll use SQLX_OFFLINE=true to run jive-api unit tests. This ensures that the tests don't require an active database connection and can run in any environment. Also, we will ensure that this unit test job runs as fast as possible, so we will optimize unit tests where possible.

Relaxing the Gates: Advisory Checks

Besides adding a fast unit test job, we're going to relax some of the existing checks, turning them into advisory gates instead of required ones. This will ease the burden on developers during the PR process and give them more flexibility while they're working on their code.

Flutter Analyzer as Advisory

One of the checks we're going to relax is the Flutter analyzer. Right now, any warnings from the Flutter analyzer will fail a PR. But, Flutter can sometimes be a bit chatty with warnings. Before the Flutter analyzer can be set to be advisory, we will need to clean up its current warnings. These warnings, while useful, can sometimes block PRs for minor issues or stylistic preferences. We will make the Flutter analyzer advisory until the warnings are cleaned up. This allows developers to focus on fixing the critical issues first and allows a more gradual improvement in the code's style. We will also prioritize cleaning up all Flutter analyzer warnings in the future.

Benefits of Advisory Checks

  • More Flexibility: Developers can submit PRs more easily, even if some of the advisory checks fail. This allows them to iterate faster and get feedback from other reviewers earlier in the process.
  • Prioritization: Developers can focus on the required checks (like the new unit tests) and prioritize addressing the issues highlighted by advisory checks later.
  • Reduced Noise: Fewer PR failures due to non-critical issues, leading to less distraction and a better development experience.

Documentation and Acceptance Criteria

To ensure this change is successful and easy to understand, we'll need a few things in place:

Acceptance Criteria

  • Fast Job: The new unit-only job must run in under 3 minutes. This is crucial for providing quick feedback to developers.
  • Required Status: The new unit-only job must be required. This ensures that the essential checks pass before a PR can be merged.

Documentation in CONTRIBUTING

We will update the CONTRIBUTING.md file to reflect these changes. This will include:

  • A description of the new unit-only job and its purpose.
  • Information about which checks are now advisory and why.
  • Instructions on how to address failures from advisory checks.

This documentation ensures that everyone on the team understands the new CI process and how to work with it effectively. Clear, concise documentation is crucial for onboarding new developers and keeping everyone on the same page.

The Big Picture: Why This Matters

So, what's the big deal? Why are we going through all this trouble? Well, improving our CI pipeline is all about improving the overall development experience. By adding the fast unit-only job, we're creating a feedback loop that's faster and more efficient. Developers get immediate feedback on their code, allowing them to catch and fix issues earlier in the process. This leads to fewer bugs, less time wasted on debugging, and ultimately, a more stable and reliable product.

Relaxing some of the advisory checks gives developers more flexibility and reduces the chances of their PRs being blocked by minor issues. This allows them to focus on the essential work and iterate faster. A streamlined CI process makes the development workflow easier and more enjoyable. These changes are a win-win for everyone involved in the project. Everyone wins, including our users, and helps us build the best product possible. By keeping our pipeline fast, we make development faster.

Next Steps

Here's what we need to do to make this happen:

  1. Implement the Unit Test Job: We need to add the new CI job that runs the unit tests for jive-api and jive-core. This includes setting up the necessary configuration and ensuring the tests run quickly.
  2. Adjust the Security Scan Gate: Set the security scan job as required to protect code quality.
  3. Update CONTRIBUTING.md: We need to update the CONTRIBUTING.md to reflect these changes. This involves documenting the new unit-only job and which checks are now advisory.
  4. Monitor and Iterate: We will monitor the performance of the new CI pipeline and make adjustments as needed. This is an ongoing process.

We have to keep in mind, that these changes are designed to improve efficiency, and make development more enjoyable for everyone involved in this project. So, let's get to work! If you have questions or ideas, don't hesitate to jump in! Together, we can build a stronger, faster, and more efficient CI pipeline and create an even better development experience.