Salt 0.17.5 Release Notes

Version 0.17.5 is another bugfix release for 0.17.0. The changes include:

  • Fix user.present states with non-string fullname (9085)
  • Fix virt.init return value on failure (6870)
  • Fix reporting of file.blockreplace state when test=True
  • Fix network.interfaces when used in cron (7990)
  • Fix bug in pkgrepo when switching to/from mirrorlist-based repo def (9121)
  • Fix infinite recursion when cache file is corrupted
  • Add checking for rev and mirror/bare args in git.latest (9107)
  • Add alias (points to cmd.wait) (8612)
  • Fix stacktrace when prereq is not formed as a list (8235)
  • Fix stdin issue with lvdisplay command (9128)
  • Add pre-check function for range matcher (9236)
  • Add exception handling for psutil for processes that go missing (9274)
  • Allow _in requisites to match both on ID and name (9061)
  • Fix multiple client timeout issues (7157 and 9302, probably others)
  • Fix ZMQError: Operation cannot be accomplished in current state errors (6306)
  • Multiple optimization in minion auth routines
  • Clarify logs for minion ID caching

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed

Tagged ,

Salt 0.17.0 Release Notes

The SaltStack team has just released 0.17.0, which happens to be the biggest release in their history. I'm very excited about the new features in this release and I'm excited to start playing with it. See below for the official release notes.

The 0.17.0 release is a very exciting release of Salt, this brings to Salt some very powerful new features and advances. The advances range from the state system to the test suite, covering new transport capabilities and making states easier and more powerful, to extending Salt Virt and much more!

The 0.17.0 release will also be the last release of Salt to follow the old 0.XX.X numbering system, the next release of Salt will change the numbering to be date based following this format:


So if the release happens in November of 2013 the number will be 13.11.0, the first bugfix release will be 13.11.1 and so forth.

Major Features


The new Halite web GUI is now available, a great deal of work has been put into Halite to make it fully event driven and amazingly fast. The Halite UI can be started from within the Salt Master, or standalone, and does not require an external database to run, it is very lightweight!

This initial release of Halite is primarily the framework for the UI and the communication systems making it easy to extend and build the UI up. It presently supports watching the event bus and firing commands over Salt.

Halite is, like the rest of Salt, Open Source!

Much more will be coming in the future of Halite!

Salt SSH

The new salt-ssh command has been added to Salt. This system allows for remote execution and states to be run over ssh. The benefit here being, that salt can run relying only on the ssh agent, rather than requiring a minion to be deployed.

The salt-ssh system runs states in a compatible way as Salt and states created and run with salt-ssh can be moved over to a standard salt deployment without modification.

Since this is the initial release of salt-ssh, there is plenty of room for improvement, but it is fully operational, not just a bootstrap tool.


Salt is designed to have the minions be aware of the master and the master does not need to be aware of the location of the minions. The new salt roster system was created and designed to facilitate listing the targets for salt-ssh.

The roster system, like most of Salt, is a plugin system, allowing for the list of systems to target to be derived from any pluggable backend. The rosters shipping with 0.17.0 are flat and scan. Flat is a file which is read in via the salt render system and the scan roster does simple network scanning to discover ssh servers.

State Auto Order

This is a major change in how states are evaluated in Salt. State Auto Order is a new feature that makes states get evaluated and executed in the order in which they are defined in the sls file. This feature makes it very easy to see the finite order in which things will be executed, making Salt now, fully imperative AND fully declarative.

The requisite system still takes precedence over the order in which states are defined, so no existing states should break with this change. But this new feature can be turned off by setting state_auto_order: False in the master config, thus reverting to the old lexicographical order.

state.sls Runner

The state.sls runner has been created to allow for a more powerful system for orchestrating state runs and function calls across the salt minions. This new system uses the state system for organizing executions.

This allows for states to be defined that are executed on the master to call states on minions via salt-run state.sls.

Salt Thin

Salt Thin is an exciting new component of Salt, this is the ability to execute Salt routines without any transport mechanisms installed, it is a pure python subset of Salt.

Salt Thin does not have any networking capability, but can be dropped into any system with Python installed and then salt-call can be called directly. The Salt Thin system, is used by the salt-ssh command, but can still be used to just drop salt somewhere for easy use.

Event Namespacing

Events have been updated to be much more flexible. The tags in events have all been namespaced allowing easier tracking of event names.

Mercurial Fileserver Backend

The popular git fileserver backend has been joined by the mercurial fileserver backend, allowing the state tree to be managed entirely via mercurial.

External Logging Handlers

The external logging handler system allows for Salt to directly hook into any external logging system. Currently supported are sentry and logstash.

Jenkins Testing

The testing systems in Salt have been greatly enhanced, tests for salt are now executed, via, across many supported platforms. Jenkins calls out to salt-cloud to create virtual machines on Rackspace, then the minion on the virtual machine checks into the master running on Jenkins where a state run is executed that sets up the minion to run tests and executes the test suite.

This now automates the sequence of running platform tests and allows for continuous destructive tests to be run.

Salt Testing Project

The testing libraries for salt have been moved out of the main salt code base and into a standalone codebase. This has been done to ease the use of the testing systems being used in salt based projects other than Salt itself.

StormPath External Authentication

The external auth system now supports the fantastic Stormpath cloud based authentication system.

LXC Support

Extensive additions have been added to Salt for LXC support. This included the backend libs for managing LXC containers. Addition into the salt-virt system is still in the works.

Mac OS X User/Group Support

Salt is now able to manage users and groups on Minions running Mac OS X. However, at this time user passwords cannot be managed.

Django ORM External Pillar

Pillar data can now be derived from Django managed databases.

Fixes from RC to release

  • Multiple documentation fixes
  • Add multiple source files + templating for file.append (#6905)
  • Support sysctl configuration files in systemd>=207 (#7351)
  • Add and file.replace
  • Fix cross-calling execution functions in provider overrides
  • Fix locale override for postgres (#4543)
  • Fix Raspbian identification for service/pkg support (#7371)
  • Fix cp.push file corruption (#6495)
  • Fix ALT Linux password hash specification (#3474)
  • Multiple salt-ssh-related fixes and improvements

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed

Tagged ,

Salt Air 17

The SaltStack team has published yet another Salt Air broadcast just two days after the last. This episode outlines many of the new features in the 0.17.0 release including:

  • salt-ssh, a new ssh-based transport mechanism.
  • halite, the Salt Web-UI.
  • State Auto Ordering, states are now fully imperative and declarative!
  • State runner, an upgrade from the overstate system
  • Salt thin, the ability to run Salt without a network transport
  • much, much more..

Check out Salt Air 17 on YouTube. (20:05)

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed


Install and configure Halite (alpha) on Arch Linux

Halite, the Salt Web UI, was recently released and I had a chance this morning to sit down and play with it. So far I'm very impressed with the functionality and with how lightweight it is. In this post I want to outline how to install and configure Halite and begin using Salt via the Web UI.


At the time of this writing Halite is very new and packages aren't yet available for every distribution. Based on that fact these instructions are specific to Arch Linux (I just happen to be the package maintainer for Salt and Halite). If you're following these instructions on a different distribution, as always, your mileage may vary.

Also, these instructions all take place on the Salt Master server. Halite can, apparently, run in a different configuration, but that is beyond the scope of this post.

Lastly, these instructions are a work in progress. If you can help improve this document, please leave a comment.


The Halite package is available in the Arch User Repository (AUR). It relies on Salt 0.17.x or above.

yaourt -S halite


Halite requires two core items be configured before you can successfully begin using it. First, you'll need to configure the user(s) that will have access, and what level of access they'll have. Second, you'll need to configure Halite itself. This primarily consists of configuring the IP/port and SSL certificate information. I'll outline each of these below.


Halite ties into the external_auth system in order to authenticate users. This means you can use existing authentication mechanisms to manage your Halite users. In this example we'll use the pam module, which means users authenticating to Halite will use their system account and password as exists on the Salt Master.

Edit your /etc/salt/master config file and look for the external_auth section. Alter the example, or append just below it, something like this:

          - .*
          - '@runner'
          - '@wheel'

There are two items within the above example that you'll want to change. First, update myusurname with your system username. Second you can limit the modules that the user is able to run in the line just below. The example - .* allows the user to run all modules. Examples of limiting to specific modules are shown in the example external_auth section of the config file.

Repeat these definitions for any users that you want to grant access to Halite.

Generating certs

Halite can run encrypted or unencrypted. This section will outline how to use Salt itself to generate the CA, CSR and cert that Halite will use to ensure encrypted connections. If you're not concerned with the encryption of your installation, or if you already have generated certificates, you can skip this section.

Note: Use of the tls module within Salt requires the pyopenssl python extension. If you do not have this installed you'll need to install it and restart the minion generating the certificate. (On arch this is the python2-pyopenssl package).

First we'll generate the CA. This is done using the tls module:

salt '*' tls.create_ca salt

Second, we'll generate the CSR:

salt '*' tls.create_csr salt

Finally we'll create the certificate:

salt '*' tls.create_ca_signed_cert salt localhost

These commands will generate localhost.{crt,csr,key} in /etc/pki/salt/certs/. We'll then use these files in our Halite configuration in order to enable SSL.

For more information on using the tls module, see the salt.module.tls documentation.


We're almost finished with the installation at this point. We simply need to define our Halite core configuration and we'll be ready to start using it! This is done by appending, inside your /etc/salt/master config, the following:

    level: 'debug'
    server: 'cherrypy'
    host: ''
    port: '8080'
    cors: False
    tls: True
    certpath: '/etc/pki/salt/certs/localhost.crt'
    keypath: '/etc/pki/salt/certs/localhost.key'


You'll need to restart your Salt Master for this new config to take effect. Once you've done that you should be able to connect to the Salt Master on port 8080 and see the new Halite Web UI.

In a future post maybe I'll outline usage of the UI and outline some of the features, but until then I'll let you explore on your own. Good luck, and happy Salting!

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed


Salt Air 16 : Salt SSH

Today the SaltStack team published a new Salt Air broadcast "Salt SSH". This episode demonstrates the new salt-ssh functionality in the upcoming 0.17.0. The new salt-ssh allows you to connect to any number of machines using SSH as a transport. This method doesn't require a minion to be installed on the client machine.

In the broadcast they spin up a 100 node riak cluster and demonstrate the time it takes over the SSH transport. If you're at all interested in the new salt-ssh I recommend checking out Salt Air 16 on YouTube.

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed


SaltConf 2014 Call for Speakers

Below is a copy of the announcement for SaltConf 2014, the SaltStack user conference to be held Jan 28-30, 2014.

Don't miss this opportunity to contribute to something big and speak at SaltConf 2014. SaltConf is the annual SaltStack user conference that will run January 28 - 30, 2014 in Salt Lake City. Deadline for speaker submissions is Oct. 21, 2013. Your submission doesn't have to be perfect, if selected we'll refine it with you. Make sure to get your best ideas in early because space is limited and we'll give preference to early submissions. Speakers will receive a complimentary SaltConf pass if their talk is accepted.

Session topics we are looking for include...

Real-world use cases of SaltStack:
  • DevOps
  • Private cloud building
  • Public and hybrid cloud orchestration
  • Enterprise systems management
  • Configuration management
  • Unique examples of data center automation
  • Management of open and proprietary technologies
  • Virtualization management
  • Benchmarks
  • Complex, large-scale deployments or implementations
  • Advanced SaltStack topology, architecture, configuration
  • Hacking and developing Salt
  • SaltStack formula and states deep dive
  • Python, YAML, Jinja
  • Returners, renderers, reactors
  • Integrations
  • SaltStack commands and modules
  • Salt Cloud, Salt Virt
  • Salt SSH
  • Extending SaltStack

Abstracts can be submitted here or via email to

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed


Vote for SaltStack in Linux Journal

Linux Journal has posted their Readers' Choice Awards for 2013 and it's time to vote! SaltStack is nominated in five categories and Tom Hatch is nominated in one. Please take a few minutes and go show your support for the project by voting for SaltStack!

Edit: Looks like I'm late to the game.. voting ends today!

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed


Salt 0.16.4 Release Notes

Version 0.16.4 is another bugfix release for 0.16.0, likely to be the last before 0.17.0 is released.

The changes include:

  • Multiple documentation improvements/additions
  • Added the osfinger and osarch grains
  • Properly handle 32-bit packages for debian32 on x86_64 (6607)
  • Fix regression in yum package installation in CentOS 5 (6677)
  • Fix bug in hg.latest state that would erroneously delete directories (6661)
  • Fix bug related to pid not existing for (6679)
  • Fix regression in MySQL returner (6695)
  • Fix IP addresses grains (ipv4 and ipv6) to include all addresses (6656)
  • Fix regression preventing authenticated FTP (6733)
  • Fix setting password for windows users (6824)
  • Fix file.contains on values YAML parses as non-string (6817)
  • Fix file.get_gid, file.get_uid, and file.chown for broken symlinks (6826)
  • Fix comment for service reloads in service state (6851)

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed

Tagged ,

How to use require and watch statements

The configuration management system in SaltStack supports many powerful options. From the simple packge installation to the use of templating and conditional statements, SaltStack States can vary from trivial to complex. Fortunately there is a robust method of defining dependencies between States allowing you to define the required order of operations. This article will outline how to use the require, require_in, watch, and watch_in definitions.


In the SaltStack world requisites come in two types. Direct requisites and "requisite_ins". These requisites are directional, allowing you to essentially say "I depend on something" or "something depends on me".


This is an example of using the require statement:


    - source: salt://edit/vimrc
    - require:
      - pkg: vim

So in this example, the /etc/vimrc file will not be placed (managed) until the vim package is installed.


This is the same example, this time using require_in.

    - require_in:
      - file: /etc/vimrc

    - source: salt://edit/vimrc

This example is essentially the same, except the vim package is saying /etc/vimrc depends on me.

In the end, a single dependency map is created and everything is executed in a finite and predictable order.

The same can be done with the watch and watch_in statements.


This is an example of using the watch statement. In this case the running service, ntpd, will watch for changes in the /etc/ntp.conf file and restart if changes are made.

    - watch:
      - file: /etc/ntp.conf

    - source: salt://ntp/files/ntp.conf


In this example the /etc/ntp.conf file is declaring that it should be watched by the ntpd service.


    - source: salt://ntp/files/ntp.conf
    - watch_in:
      - service: ntpd


The require, require_in, watch and watch_in statements are very powerful tools that allow you to define the dependencies between items in your State formulas. Whether a service should watch a file for changes, or a package be required before a service can be started, these declarations allow you to define the logical order of execution within your States.

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed

Tagged , , ,

SaltStack State Formula Explained

I've written a few other posts about SaltStack Configuration Management (1, 2, and 3), but I realize I didn't spend much time on writing the actual formula. This post will try to explain how these formula are laid out, where they are stored, and the syntax requirements for proper parsing.

The Layout

The location of your State formulas is defined in the Master config file using the file_roots definition. The default on Linux is /srv/salt and on FreeBSD systems it is /usr/local/etc/salt/states. As you begin to write State formulas you'll need to store them in the appropriate path as defined on your system.

As you begin to populate this directory with State formula you'll end up creating sub-directories within your file_roots to store them. These directories should generally be named after the formula it will represent. Common directories are nginx, users, postgresql, etc. Each of these sub-directories will hold State formula instructions on how to install nginx, setup users, or setup the Postgresql database (for example).

An example file_roots tree:


This example illustrates three different ways to layout your formulas. These formula can then be targeted based on their name or path.

salt '*' state.sls vim [applies the vim.sls formula]
salt 'web1*' state.sls nginx [applies the nginx/init.sls formula]
salt 'web1*' state.sls mysql.client [recieves the mysql/client.sls]
salt 'db1*' state.sls mysql.server [applies the mysql/server.sls]

If you specify a State formula using the state.sls syntax, SaltStack will look for either a .sls of that name, or an init.sls within a folder of that name. We can also specify role-specific formula as in the MySQL example above. When defining mysql.client as the requested State formula we're targeting the client.sls file within the mysql directory.

The Syntax

SaltStack State formulas use the yaml markup language for their syntax. Unlike some other projects, it is not a domain specific language (DSL), but a standard markup language used by a number of projects. The syntax is very simple, but I find that new users still stumble when first writing these formulas. The number one thing to remember is that spacing matters. Yaml is all about space-indented blocks that are then parsed to represent values, lists, etc.

An example formula to install and start the Nginx web server:

    - installed
    - running
    - require:
      - pkg: nginx

The key thing to point out here is that there are always two spaces between one level to the next. You'll want to avoid tabs or other levels of indentation. Two spaces, that's all you need.

As a tip, you might want to check out my previous post on SaltStack State Vim Syntax Highlighting to help ensure your syntax is always correct.

Additional Reading

If you haven't already, you'll want to check out the Getting Started With States official documentation. Hopefully, between this post and that, you'll be writing valid State formula in no time!

Follow on Facebook | Follow on Google+ | Follow on Twitter | RSS Feed