Liberty cycle retrospective in Puppet OpenStack

news

Author: Emilien
Source: Planet OpenStack

Things are moving very fast in OpenStack; it might be useful to take a short break and write down a little bit of retrospective; it will help to see what happened in Puppet OpenStack project during the last months.

Before going through details, let’s remind us what were our plans for Liberty cycle, and understand what we wanted to achieve in six months.
For that, I suggest you to read the blog post I wrote at the end of Liberty Summit. I’ll take inspiration from the plan to write the retrospective.

 

Releases and master branch policy

The blueprint has been approved and you can read details here.

Here is a summary of our decision:

  • Accept that master branch is not supposed to work on stable releases of OpenStack.
  • Functional testing CI jobs should pass using the latest testing package repositories from the Ubuntu and RHEL distributions.
  • Submit a feature in master only if it can be tested by functional testing CI jobs. We make an announcement on the mailing-list each time we update the repositories for the functional tests*.
  • Help developers to easily backport patches to stable branches when needed**.

* We had (and are having) hard time to use latest testing package repositories, because none of them provide real CI that deploys all OpenStack components like we are doing.
We had some CI downtimes because of broken upstream packaging and it took a lot of time to unblock CI, and debug what was going wrong.
Most of the time, it was due to new OpenStack requirements (new version or new dependency) and packaging had hard time to follow all of them quickly without automation.
A first and easy solution would be to wait a little bit more before bumping our manifests to testing repositories. But we can find alternatives (like experimental jobs) to be more effective.
The downside of staying too much time on stable/* branches for master is we can’t really test new features that are implemented in OpenStack.
That’s one of the reasons RDO trunk & Ubuntu Liberty staging allowed us to do it, since we could install latest version of OpenStack (unstable though).
So I suggest we talk about this during the next summit and see possible alternatives that would satisfy our next development cycle.

** Automatic cherry-picks has been rejected by OpenStack Community until now and I don’t think it will land one day. That means we have to continue to evangelize our backport policy and make sure our documentation correctly defines it.

 

Puppet module compliance

Puppet OpenStack is growing fast and every cycle we have to deal with new OpenStack projects and new Puppet modules that come with. It’s a real challenge to keep the code consistent, tested, and documented so we took actions:

  • Document how to welcome new Puppet modules.
  • Use puppet-openstack-cookiebutter to keep our basic structure consistent, and synchronize our modules using puppet-modulesync.
    We wanted to automate the task with Gerrit but the discussion but it has also been rejected by OpenStack Infra so for now, we will continue to manually synchronize our modules when a new big change happens.
  • getting some modules approved by Puppet Labs and this is happening.

All this work will really help our project to scale-up and continue to manage more and more modules.

 

Handle default OpenStack parameters values

So until now, when adding new parameters in our Puppet modules, we strictly defaulted them to the same default value in OpenStack to keep consistent with upstream.
Since then, we’ve been working on the *_config providers so we will use “<service default=”DEFAULT”>” instead of ‘undef’ or ‘false’ for default values. If “</service><service default=”DEFAULT”>” is set, the Puppet provider will make sure the parameter is absent from OpenStack configuration file, so the service will run with the default parameter defined in OpenStack code.

That way, we will stop to worry about upstream default parameters changes and rely 100% on upstream configuration, with the capability to change it in our interface.

 

Continuous Integration

Liberty cycle was clearly a huge milestone for Puppet OpenStack CI. Since the things are moving a bit fast, we try to keep the documentation updated so people can understand what we’re testing.

The biggest thing regarding CI is functional testing.
We now have two kind of jobs:

  • beaker: they allow any developer to test our Puppet modules in their development environment.
    Note that OpenStack Infra also decided to use Beaker for functional testing, which means Beaker is now widely used in OpenStack community for Puppet module testing.
  • puppet-openstack-integration: we initiated a new functional testing job that aims to validate an OpenStack Cloud can be deployed with the modules and tested by Tempest. If you’re interested to learn more about it, I suggest to have a look at this presentation.

In the next cycle, I hope we can continue to move forward with other modules and add more components & tests.
We are also investigating multi-node jobs but this is really work in progress now, we might have updates after the Summit.

Having functional testing made our modules stronger, and we fixed a lot of issues we had with orchestration.
Also, you can now create dependencies between patches from different modules, and you’ll be able to test the final result, thanks to Beaker & Integration jobs that use zuul-cloner to prepare Puppet environment.
The efforts that we spent (and are spending) on CI can be seen in the quality and adoption of our modules; as a retrospective I would say it something we should have done earlier.

 

No puppet-oslo

So we decided to not push puppet-oslo module upstream and keep patching our modules asynchronously, since OpenStack project are not always consistent with Oslo projects versions.
I haven’t heard about any bad feedback so I guess we can continue that way for the next cycle.

 

Until now, we talked about our Liberty plans. Now, let’s see other topics that happened during the last months.

 

Collaboration request with Fuel folks

So early during the cycle, we initiated communication between Puppet OpenStack & Fuel groups.
We broke the ice (or iceberg) and I think we did it. So far we had productive discussions and I’m more than happy this is happening.

  • Fuel developers are changing they way to consume Puppet OpenStack modules.
  • They are now reporting bugs on the right place, and give huge help on bug triage.
  • Code contributions are happening more than frequently and some folks are doing good reviews.
  • They documented the way Fuel should deal with Puppet upstream modules.

Thanks a lot for keeping your mind open and I hope it will continue that way in the future. We have a lot of learn from each others and I’m looking forward to seeing the progress being made from both groups.

 

Sprint

We recently did our first development sprint!
Feel free to look at the etherpad to see what we achieved in three days.
With the goal to improve the future sprints, we are currently doing a Sprint retrospective.
I hope you enjoyed the sprint and that the next one will be even better.

 

6.0.0 and 5.1.0 were released

We shipped the first Kilo release (6.0.0) and a second Juno version (5.1.0) during early July.
I think this release was a bit late but we have to take in account we wanted Keystone v3 support as highest priority and we made it.
Hopefully we will release stable/liberty (7.0.0) in the same time as Ubuntu & RDO will provide stable packaging.

 

OpenStack Summit is coming next month, and we’re actively working on it. Feel free to bring any suggestion in the etherpad concerning Liberty retrospective.

I hope this post helped you to understand what our group did.
It’s a real pleasure to lead this project, and I’m looking forward to seeing you all in Tokyo.</service>

Powered by WPeMatico