Mar 27, 2014

Horizon is translated to German for OpenStack Icehouse!

After my call for help two days ago, a few new people started helping with the German translations and we've achieved the goal of translating 100 per cent - that's all 2141 strings!
This was done by Christian Berendt, Marita Werner, Robert Simai, Sascha Peilicke, User "Schwefel", User "transistor" and myself! Thanks for getting this done!

I'm impressed that we got this final step done so quickly!

A special applause to Marita for doing the final 60 strings - I consider the last the hardest ones since those are the ones that we all skipped ;)

Mar 25, 2014

Translating OpenStack Dashboard (Horizon) - Let's get German in!

The OpenStack 18n team has recently asked for translations of the dashboard for OpenStack Icehouse, here're two emails with details (first mail, second mail).

My friend Robert suggested to do the German translations to finally have the dashboard available in German as well - and Sascha stepped up to help coordinating the German language team.

For German, there's only one resource left to translate called "openstack-dashboard-translations" - and it's available like all other OpenStack translations at transifex.


If you want to join, just do the following (the instructions apply for other languages as well, some links go directly to the German parts):
  1. Join Transifex at http://transifex.com/.
  2. Join the German OpenStack translation team and wait until your request is accepted.
  3. Go to  the German Horizon page
  4. Click on "Openstack Dashboard Translations" and start translating.

A few days ago there were 860 untranslated strings for German, now we're down to 525. Who's going to help to get it down to zero until the deadline (I've heard 4th of April)?

If you want to learn more about the OpenStack i18n team, visit this wiki page.

Jan 24, 2014

Python Virtualenv awesomeness - and developing openstack-doc-tools

Testing and developing the openstack-doc-tools package, I really came to love the python virtualenv package. The package allows installation of python packages in a local environment (virtualenv = virtual environment) for usage.

Let me guide you on how I use it for openstack-doc-tools - the repo the OpenStack documentation team uses for its tools.

First I change to the checked out openstack-doc-tools repository (get it from https://github.com/openstack/openstack-doc-tools).

Then, let's setup the virtualenv: I could setup a virtualenv manually using for example "virtualenv myenv" and install packages. But the repository knows already about virtualenv and thus I can run "python tools/install_venv.py" and it installs a virtual python environment named '.venv' using the directory '.venv' of openstack-doc-tools. The requirements for the project as mentioned in the requirements.txt and test-requirements.txt files, will get installed into this virtualenv.

Once the command finishes, it outputs a nice usage information, and I follow the option to "source .venv/bin/activate" from my shell to run commands in the virtualenv.

Next, I need to build the package via "python setup.py build" and then install the package with "python setup.py install".

Now I can easily execute commands for validating repositories. So, I can change my working directory to the api-site repository and run for example the command:
"openstack-doc-test  --api-site --check-niceness --force"

to test that the recent changes to the niceness check work fine on all files in the repository.

Jan 19, 2014

Setting up gating in the OpenStack intrastructure

One thing that impressed me with OpenStack, is the common gerrit workflow that is not only used for code but also for documentation and the infrastructure.
As I wrote in my last blog post, I enabled further validation of the documentation.
This article will describe how the validation has been done using the OpenStack CI infrastructure.

The validation is done via Jenkins and to execute a process during review, as part of validation after it is approved, after it is merged or at periodic times, you need to create jobs for Jenkins and schedule them via Zuul, the OpenStack project gating system. Let's look at an example how this can be setup.

An example of gating

Let's look at how I setup gating for the api-sites projects. The patch I sent is https://review.openstack.org/#/c/64795/ and will be used as example in this blog post.
First, you should learn about the OpenStack Continous Integration infrastructure which is documented at http://ci.openstack.org/.  Then you should check out the infra repository called config since it contains all configuration files you need to modify. Instead of directly editing Jenkins configuration, YAML files are written that the Jenkins Job Builder then uses to create Jenkins jobs.

Defining a job

For a simple job, you can just specify it. With openstack-doc-tools, I wanted to reuse the same job not only for the api-sites but for all other projects as well. This can be done with a job template and those can be grouped as job-groups. Since my jobs can be executed by tox, I could reuse the 'gate-{name}-tox-{envlist}' template (file modules/openstack_project/files/jenkins_job_builder/config/python-jobs.yaml). Parameters are specified in curly braces, thus it needs as parameters the "name" of the job and the "envlist" that tox will run - basically via "tox -e envlist".
To save typing, I used job-groups to define all four jobs in one places in the file modules/openstack_project/files/jenkins_job_builder/config/manuals-jobs.yaml:
 - job-group:
    name: openstack-doc-jobs
    jobs:
      - gate-{name}-tox-{envlist}:
          envlist: checkniceness
      - gate-{name}-tox-{envlist}:
          envlist: checksyntax
      - gate-{name}-tox-{envlist}:
          envlist: checkdeletions
      - gate-{name}-tox-{envlist}:
          envlist: checkbuild

So, this creates 'gate-{name}-tox-checkniceness' etc, the variable envlist gets substituted with checkniceness for the first gate etc.

Now, to use this job-group, I added to modules/openstack_project/files/jenkins_job_builder/config/projects.yaml a project definition:
- project:
    name: api-site
    github-org: openstack
    node: precise

    jobs:
      - openstack-doc-jobs

The name parameter in the job-group is now substituted with "api-site", thus this creates the four jobs:
gate-api-site-tox-checkniceness
gate-api-site-tox-checksyntax
gate-api-site-tox-checkdeletions
gate-api-site-tox-checkbuild

Now, the created jobs are ready to be run at specific events.

Gating jobs

The OpenStack CI infrastructure uses a tool called Zuul to launch jobs in response to specific gerrit events. The configuration happens via a central YAML file, it is modules/openstack_project/files/zuul/layout.yaml.
Since I like to use the same jobs for several projects, I made use of a very recent feature of Zuul called project-template. I created a template with the name "openstack-doc-gate":
project-templates:
...
  - name: openstack-doc-gate
    check:
      - gate-{name}-tox-checkniceness
      - gate-{name}-tox-checksyntax
      - gate-{name}-tox-checkdeletions
      - gate-{name}-tox-checkbuild
    gate:
      - gate-{name}-tox-checkniceness
      - gate-{name}-tox-checksyntax
      - gate-{name}-tox-checkdeletions
      - gate-{name}-tox-checkbuild

The jobs are setup to be run for each review (check) and once a patch has been approved (gate).

This template is then used in the configuration of "api-site". The name here is the name of the git repository (api-site in project openstack) and the last component (api-site) is passed as parameter to the template which results in the four jobs created in projects.yaml:
  - name: openstack/api-site
    template:
      - name: openstack-doc-gate
    post:
      - openstack-api-quick-start
      - openstack-api-site
      - openstack-api-ref

The post jobs are unchanged, these are the jobs that will be run after a patch is merged.

Additionally, I marked the checkniceness job as non-voting:
  - name: gate-api-site-tox-checkniceness
    voting: false

As mentioned in my previous post, the patch has been merged and is running just fine.
If you want to see Zuul in action, check its great status page.

Further work

I've written a followup patch to gate the API projects and operations-guide repository, it is currently up for review at https://review.openstack.org/#/c/64795/. Once this is in, the openstack-manuals can get similiar treatment. I've decided to use a separate patch for easier review since the openstack-manuals patch will remove some existing configuration.
Writting this up, I noticed that the checkniceness job is run also as non-voting gating job. This is not needed, so I wrote a simple patch to remove it: https://review.openstack.org/67702.

Jan 17, 2014

Validating OpenStack API projects

The OpenStack documentation team has validated the manuals in the openstack-manuals repository using a python tool. We now have moved this and other tools into a new openstack-doc-tools repository. The goal is to have a single place for the tools that are needed for different documentation repositories.

During the last weeks, I expanded the tools so that they handle also the API project repositories like api-sites and volume-api (the published versions are found via http://api.openstack.org/). The tools found many problems in the current API projects, including some projects that did not even built. The errors found by these are a clear sign that we do need these gating jobs. Diane Fleming and myself fixed the problems that the validation tool found. A small number of problems are still being worked on but all repositories build again and are setup for using the new gates.
Since yesterday, the api-sites repository uses this new tool for gating!

Validating your changes

If you like to run the gates locally: Install the python tox package and run 'tox' from the top-level directory to use the same tests that are done as part of our Jenkins gating jobs. If you like to run individual tests, run:
  • 'tox -e checkniceness' - to run the niceness tests
  • 'tox -e checksyntax' - to run syntax checks
  • 'tox -e checkdeletions' - to check that no deleted files are referenced
  • 'tox -e checkbuild' - to actually build the manual(s)
 Note that these checks only test the files of the most recent commit, so first 'git commit' everything, then run 'tox'. To check all files, pass '--force' as parameter to the tox command, for example 'tox -e checkniceness -- --force'. The '--' is important, it passes the option down to the validation tool.

Note that the tools need the maven package as requirement, so install that one as well.

Repositories gating

As a first step, api-sites is gating now with these four checks. The niceness check is optional (non-voting). The other api projects (compute-api, database-api, identity-api, image-api, netconn-api, object-api, and volume-api) will soon be setup as well for this.
Additionally, these gates will replace the existing gates for the openstack-manual and operations-guide repositories. Once this is done via have for all repositories the same setup and just a single repository with all the validation tools.

openstack-doc-tools repository

The repository can be downloaded as a Python package from pypi and currently contains tools for:
  • validation/gating: as explained above
  • translated documents: Tom Fifieldt is looking on enhancing and using these now.
  • Autogeneration of tables: Shaun McCance is enhancing these to nicely generate configuration tables of options specified in the various conf files like nova.conf.
  • cleanup tools: Those can be used to remove some whitespace.
Help in improving the tools is always welcome. If you have questions, join the documentation team on the #openstack-doc IRC channel or on the openstack-docs mailing list

Thanks

Thanks especially to Fungi and Clark for guidance and review on how to do this in the best way, to Diane for fixing many problems, for Tom, Anne and others for reviews and comments!

Nov 16, 2013

SUSECon '13 - Final day

It's Friday - the final day of SUSECon. The closing keynote was by Lew Tucker from Cisco who's also the OpenStack Vice Chairperson.

He gave a great overview of cloud computing and OpenStack - and how Cisco plays a role in it. He also cited some impressive numbers on network traffic world wide.
It was interesting to see that Cisco UCS has an API that you can use to manage the Cisco hardware.
Also a demo was shown that showed the integration of Cisco UCS together with SUSE Cloud - and SUSE Manager and SUSE Studio for a complete lifecyle management.

At the end of the keynote the video "What's the Chameleon Say" was shown. SUSE's Russ Dastrup wrote a great script and filmed the  initial version  with his kids and some friends - and then during SUSECon filmed the version below with participants.

Btw. additional videos are available from the SUSEvideo channel on YouTube.



I attended then the session called "wicked trip into wicket network management". Olaf Kirch and Matthias Eckermann explained the complexities of networking on Linux and how the ifcfg framework will be replaced for SUSE Linux Enterprise 12 by a new concept called "wicked". Wicked uses a wickedd daemon and a wicked command line utility to manage complex network setup.  It handles renames of interfaces and can be configured by both config files and via dbus. The package wicked is available for installation in openSUSE 13.1 from the Open Build Service.

Thus ends my report from SUSECon '13. If you like to see more photos, check my updated gallery.

Nov 15, 2013

SUSECon '13 - third day

Thursday morning started for me with a presentation titled "Intel and OpenStack: Contributions and challenges". Krish Raghuram explained not only how Intel helps OpenStack to work great on Intel hardware but also how Intel uses OpenStack internally. For example, they have their own OpenStack private cloud where they run 1,500,000 (hope I got all the zeros right;-) VMs across data centers. Interesting was the concept of geolocation in the cloud: With trusted computing, an image can be encrypted and then will only run in certain geos, e.g. countries. If only the diagram wouldn't look that complex...

The next presentation I attended was by Udo Seidel about "Petabyte scale out Rocks - Ceph as Replacement for Openstack's Swift and Co". He first introduced ceph and explained how it's scalable, has a flexible configuration and not a single point of failure. Ceph can also be used in OpenStack as an alternative to OpenStack Object Storage (Swift), as backend for OpenStack Block Storage (cinder) and
also as storage for OpenStack Image Service (glance).  A single backend for these kinds of storage is a quite clever idea.

I listened again into our SLE 12 systems management outlook session and was happy that I had the evening off - and concluded the evening with a good time at the pool looking at the moon, swimming, sliding and talking.


Photos from the day have been added again to my gallery.