Cfengine is a suite of programs for integrated autonomic management of either individual or networked computers. It has existed a This document represents cfengine versions 3.0.0 and later, which are a radical departure from earlier versions. Cfengine 3 has been changed so as to be both a more powerful and much simpler. Cfengine 3 is not backwards compatible with the cfengine 2 configuration language, but it interoperates with cfengine 2 so that it is "run-time compatible". This means that you can change over to version 3 slowly, with low risk and at your own speed. With cfengine 3 you can install, configure and maintain computers using powerful hands-free tools. You can also integrate knowledge management and diagnosis into the processes.
Cfengine differs from most management systems in being
Cfengine 3 consists of a number of components:
Component | New | Function |
---|---|---|
cf-agent | Active agent | |
cf-execd | Scheduler | |
cf-graph | Graph data extractor | |
cf-know | * | Knowledge modelling agent |
cf-monitord | Passive monitoring agent | |
cf-promises | * | Promise validator |
cf-runAgent | Remote run agent | |
cf-serverd | Server agent | |
cf-show | Self-knowledge extractor |
The starred components are new. The daemon formally called cfenvd i previous versions of cfengine is now called cfMonitord.
Unlike previous versions of cfengine, which had no consistent model for its features, you can recognize everything in cfengine 3 from just a few concepts.
Concept | Quick description |
---|---|
Promise | A statement about the state we desire to maintain |
Promise bundles | A collection of promises |
Promise bodies | A part of a promise which details and constrains its nature |
Data types | An interpretation of a scalar value: string, integer or real number |
Variables | An association of the form "LVALUE represents RVALUE", where rval may be a scalar value or a list of scalar values |
Functions | Built-in parameterized rvalues |
Classes | Cfengine's boolean classifiers that describe context |
If you have used cfengine before then the most visible part of cfengine 3 will be its new language interface. Although it has been clear for a long time that the organically grown language used in cfengines 1 and 2 developed many problems, it was not immediately clear exactly what would be better. It has taken years of research to simplify the successful features of cfengine to a single overarching model. To understand the new cfengine, it is best to set aside any preconceptions about what cfengine is today. Cfengine 3 is a genuine "next generation" effort, which is will be a springboard into the future of system management.
Many attempts at improving the user interface of cfengine have been proposed but none of them have been sufficiently impressive to make the change worthwhile before now. Some have gone in for an Object Oriented approach, but this imposes a hierarchical model that does not fit cfengine's autonomous peer model. The main goal in changing the language is to simplify and improve the robustness and functionality without sacrificing the basic freedoms and concepts. Concepts such as explicit loops and and tests have long been banished from cfengine and proposals to reintroduce them have been dismissed --- something better is needed. The difficulty, of course is to provide a genuine simplification and improvement that is robust and lasting: this requires a deep understanding of the problem.
Cfengine 3's new language is a direct implementation of a model developed at Oslo University College over the past four years, known colloquially as "promise theory". Promises were originally introduced by Mark Burgess as a way to talk about cfengine's model of autonomy and have since become a powerful way of modelling cooperative systems. Cfengine 3 is a generic implementation of the language of promises that allows all of the aspects of configuration management to be unified under a single umbrella.
Why talk about promises instead of simply talking about changes? After all, the trend in business and IT management today is to talk about Change Management, e.g. in the IT Infrastructure Library (ITIL) terminology. This comes from a long history of process management thinking. But we are not really interested in change -- we are interested in being in a state where we don't need to make any changes. In other words we want to be able to promise that the system is correct, verify this and only make changes if our promises are not kept.
To put it another way, cfengine is not really a change management system, it is a maintenance system. Maintenance is the process of making small changes or corrections to a model. A "model" is just another word for a template or a specification of how we want the system to work. Cfengine's model is based on the idea of promises, which means that it focuses on what is stable and lasting about a system -- not about what is changing.
This is an important philosophical shift. It means we are focused mainly on what is right and not on what is wrong. By saying what "right" is (the ideal state of our system) we are focussed on the actual behaviour. If we focus too much on the changes, i.e. the differences between now and the future, we might forget to verify that what we assume is working now in fact works.
Models that talk about change management tend to forget that after every change there is a litany of incidents during which it is necessary to repair the system or return it to its intended state. But if we know what we have promised, it is easy to verify whether the promise is kept. This means that it is the promises about how the system should be that are most important, not the actual changes that are made in order to keep them.
To familiarize yourself with cfengine 3, type or paste in the following example text:
######################################################## # # Simple test execution # ######################################################## body common control { bundlesequence => { "testbundle" }; } ######################################################## bundle agent testbundle { vars: "size" int => "46k"; "rand" int => randomint("33","$(size)"); commands: "/bin/echo" args => "Hello world - $(size)/$(rand)", contain => standard, classes => mydefine("followup","alert"); followup:: "/bin/ls" contain => standard; reports: alert:: "What happened?"; } ###################################################################### body contain standard { exec_owner => "mark"; useshell => "true"; } ###################################################################### body classes mydefine(class,alert) { on_change => { "$(class)" }; on_failure => { "$(alert)" }; }
This example shows all of the main features of cfengine: bundles, bodies, control, variables, and promises. To the casual eye it might look complex, but that is because it is explicit about all of the details. Fortunately it is easy to hide many of these details to make the example simpler without sacrificing any functionality.
The first thing to try with this example is to verify it -- did we make any mistakes? Are there any inconsistencies? To do this we use the new cfengine program cfpromises. Let's assume that you typed this into a file called test.cf in the current directory.
cfpromises -f ./test.cf
If all is well, typing this command shows no output. Try now running the command with verbose output.
cfpromises -f ./test.cf -v
Now you see a lot of information
Reference time set to Sat Aug 2 11:26:06 2008 cf3 Cfengine - 3.0.0 Free Software Foundation 1994- Donated by Mark Burgess, Oslo University College, Norway cf3 ------------------------------------------------------------------------ cf3 Host name is: atlas cf3 Operating System Type is linux cf3 Operating System Release is 2.6.22.18-0.2-default cf3 Architecture = x86_64 cf3 Using internal soft-class linux for host linux cf3 The time is now Sat Aug 2 11:26:06 2008 cf3 ------------------------------------------------------------------------ cf3 Additional hard class defined as: 64_bit cf3 Additional hard class defined as: linux_2_6_22_18_0_2_default cf3 Additional hard class defined as: linux_x86_64 cf3 Additional hard class defined as: linux_x86_64_2_6_22_18_0_2_default cf3 GNU autoconf class from compile time: compiled_on_linux_gnu cf3 Interface 1: lo cf3 Trying to locate my IPv6 address cf3 Looking for environment from cfenvd... cf3 Unable to detect environment from cfMonitord --------------------------------------------------------------------- Loading persistent classes --------------------------------------------------------------------- --------------------------------------------------------------------- Loaded persistent memory --------------------------------------------------------------------- cf3 > Parsing file ../tests/runtest_8.cf --------------------------------------------------------------------- Agent's basic classified context --------------------------------------------------------------------- Defined Classes = ( any Saturday Hr11 Min26 Min25_30 Q2 Hr11_Q2 Day2 August Yr2008 linux atlas 64_bit linux_2_6_22_18_0_2_default x86_64 linux_x86_64 linux_x86_64_2_6_22_18_0_2_default linux_x86_64_2_6_22_18_0_2_default__1_SMP_2008_06_09_13_53_20__0200 compiled_on_linux_gnu net_iface_lo ) Negated Classes = ( ) Installable classes = ( ) cf3 Wrote expansion summary to promise_output_common.html cf3 Inputs are valid
The last two lines fo this are of interest. Each time a component of cfengine 3 parses a number of promises, it summarizes the information in an HTML file called generically promise_output_component-type.html. In this case the cfpromises command represents all possible promises, by the type "common". You can view this output file in a suitable web browser to see exactly what cfengine has understood by the configuration. The output looks something like this:
Expanded promise list for common component
Constant variables in SCOPE testbundle:
Constant variables in SCOPE const:
Constant variables in SCOPE sys:
|
This is simply a summary of the `compilation'. Nothing has happened yet. To make something happen, you need to pass these promises to the agent for whom they are intended. Notice that the promise bundle has a type "agent" and that the output claims that the promises are intended for "cf-agent". In fact, each component of cfengine looks for promises that are addressed to it. (In earlier cfengine versions there were different files for each component. In cfengine 3 you can make promises for any agent in any file, the bundles label which agents will pay attention to them.)
So try verifying / keeping the promises in this file.
cf-agent -f ./test.cf cf-agent -f ./test.cf -vYou may now look also at promise_output_agent.html.
In the general case, a promise is made from one autonomous entity to another. The general theory allows for an in-depth discussion of this, but in cfengine we can often suppress the recipient of a promise and simply consider all the independent resources in the system as making promises to the system administrator or some other external entity. This is not always true, but it is an okay starting point.
"promiser" -> "promisee"The promisee is usually someone or something that is interested in the promise and will therefore verify it. For now this value is not used much in cfengine, but is present for future developments. If left unspecified it defaults to the agent that is responsible for processing the current promise-bundle (the bundle type, e.g. "agent" or "server").
We need to be able to distinguish different promises from one another and we do this by defining the promise body, which is a label on the arrow saying something about what is being promised. A promise body generally has a promise type or subject and a choice that is being specified from the set of possible things that can be promised about that subject. We write this as follows:
subject => decision/constraint/compound-attributes
Promises are usually quite complicated and will have several attributes that make up the complete promise body. e.g.
files: "/home/mark/tmp/testcopy" copy_from => mycopy("/home/mark/LapTop/words","127.0.0.1"), perms => system, depth_search => recurse("inf");
What you should notice about this example is that none of the right hand sides are actually cfengine keywords, they are all user-defined templates. Here is one example of such a template:
body copyfrom mycopy(from,server) { source => "$(from)"; servers => { "$(server)" , "failover_host" }; copy_backup => "true"; purge => "true"; }
This shows a user-defined bundle of attributes that is parameterized so that it can be re-used. In cfengine 3 you are encouraged to make libraries of such templates so enrich the expressivity of the basic language and to simplify the configuration rules.
Get used to putting quotes around all literal values in cfengine 3. This is a discipline that allows the compiler to locate errors much more easily.
Some promise attributes are `common' to any promise, e.g. transaction details. So any promise can add something of the form
action => my_actions_details;where my_action_details is the name of a body template, defined elsewhere. The action attribute described the way in which the resource behaves when it keeps its promises. e.g.
body action my_action_details { action => "warn"; ifelapsed => "10"; audit => "true"; report_level => "inform"; }
Any promise that includes action => my_actions_details will i) only warn when it is not keeping its promise, not fix the problem automatically, ii) wait until 10 minutes have elapsed before checking the promise again, iii) leave a full audit trail of its findings when verifying the promise, and iv) inform us of any changes that are made.
Body attributes that are common to all promises include:
Body lvalue | Simple/Compound | Description |
comment | (inline) | A comment string for internal documentation generation |
ifvarclass | (inline) | Not used yet. |
action | (body template) | Aspects controlling the behaviour during the promise verification transaction. |
classes | (body template) | Defining classes as a result of the outcome or promise verification. |
Making clearly motivated promises about behaviour can be a potentially complex thing to do. One way to make promises clearer in a computer science sense is to make a clean taxonomy of promise subjects or types. (The word "type" is a bit overloaded in computer science so let's try to avoid it by using the word "subject".) Cfengine 3 makes this very powerful by integrating semantic modelling into the langauge through its new knowledge agent cfKnow. But before we get into those details, it is a good idea to think about how you want to bundle aspects of management together.
Before cfengine 3, you only had the option of splitting rules into different files. Promise bundles now allow you to refer to aspects of administration in a more sophisticated manner. They also allow cfengine to give much better and more specific error reporting.
A single promise might therefore consist of many subdivisions -- or smaller promises. We use the term promise bundle (inspired by geometric fibre bundles or nerve bundles if you prefer) to describe this. In our research, we have used multiple arrows for such bundles. In the cfengine language, we simply list related promise bodies after the promiser:
bundle name and details { main_subject: context_for_promise:: "promiser object" -> "promisee" subsubject_1 => choice_1, subsubject_2 => choice_2, ... subsubject_n => choice_n; ... }This is the general form of a statement in cfengine 3. The form does not change for different subjects so it is a pattern that pervades everything. This simplification is the starting point for simpler configuration descriptions without forced assumptions.
So let's write down the cfengine language in its general form.
A promise has the following generic form:
context|classes:: "promiser object" -> "promisee" # constraints lval1 => rval1, lval2 => rval2, ... lvaln => rvaln;A simplified form that is most often used in cfengine omit the promisee, as this is usually cf-agent itself.
context|classes:: "promise object" # constraints lval1 => rval1, lval2 => rval2, ... lvaln => rvaln;Cfengine arranges promises into "bundles". Sometimes an rvalue for a given constraint might involve many attributes that are conveniently grouped together. In this case these are grouped into a "body" which is simply an aggregate of lval,rval pairs.
body name { lval1 => rval1; lval2 => rval2; }Note the placement of "," and ";" in these.
An rvalue is something on the right hand side of an assignment. The cfengine language of promises has four distiguishable meta types of object to remember. By knowing the difference between these, you will see generic patterns that make syntax easy to learn.
For example
vars: "name" slist = { "one", "2", "three" }; "float" rlist = { "1.0", "2.02", "3.33" };Cfengine validates the values before accepting them. Be careful not to confuse meta object types (string, list and function etc) with abstract types (string,int,real, etc).
$(context) $(context.name)
Although variables are protected in their contexts, you have access to variables in any context in any part of cfengine that belongs to a particular agent. Variables defined in any agent bundle can be accessed by the agent in any other agent bundle, as long as the full context is specified. If no context is specified then the present context is assumed.
Variables defined in common bodies are available to all components of cfengine. So if you want to share variables, simply define them in a common context, e.g.
bundle common v { vars: "global_1" string => "some value"; }Thereafter you can refer to this as $(v.global_1) anywhere.
The context "this" is special. It is a copy of the local context that is augmented by additional variables. For instance the variable $(this.promiser) should always point to the current object whose promised properties are being verified. The variables $(this.0), $(this.1), ... etc (or equivalently $(0), $(1)...) represent backreferences to regular expression matches.
bundle server main { vars: "list" slist => { "192.168.0.4", "myhost.example.com" }; admit: "/etc/passwd" -> @(list); "/etc/services" -> @(list); # Specific "/special/$(list)/file -> $(list); }In the latter case the file /special/192.168.0.4/file is granted to the IP address 192.168.0.4 only, and /special/myhost.example.com/file is granted to myhost.example.com only. Now suppose that we write
"/special/$(list)/file -> @(list);This now expands to
/special/192.168.0.4/file -> { "192.168.0.4", "myhost.example.com" }; /special/myhost.example.com/file -> { "192.168.0.4", "myhost.example.com" };
lval => function("arg1", $(arg2), otherfn("arg3"))
Call: access => myaccess($(actual)), Template: body files myaccess(local) { }
In cfengine 3, it is now straightforward to use the concept of a bundle to represent aspects or parts of aspects. Just as in earlier versions of cfengine, one can also place aspects in different files.
We should not think of bundles as private functions in the sense of an object programming language. Although they can have private variables, the promises within do not act on private workspace, they can act on any part of the computer system on which the agent is running. So the organization is purely formal, for ease of understanding. The organization or reorganization of promises is entirely cosmetic.
A final aspect of promise organization is authorization. Because promises given are not automatically used, unless the recipient promises to use them, one could split up aspects into files from different authorized inviduals and place access control on the possible rules in a file obtained from a given source. Thus one could deny access to certain users to make certain kinds of promises.
One of the first things you will notice in cfengine 3 is that the control section has been moved and that there is a new place to define variables, which is called "vars:". The reason for this is clarity. In cfengine 2 there is a confusion between fixed-name identifiers, e.g. Inform that set optional behaviour (the orginal meaning of control) and freely defined variables that users define and use in rules.
In cfengine 3, promises belong to bundles and attributes of promises belong in bodies. Variables male promises (to associate a name with a value) so they are placed in bundles:
vars: "identifier" type => "value";Control information is now the province of promise "bodies". A body is exactly a collection of control attributes, belonging to a promise. Apart from "control" bodies, all other bodies are tied to user defined promises. In the case of "control", these are attributes to an implicit promise by the agent, the server or the monitor, etc.
body agent control { access => { "mark", "root" }; }Then there is also a control promise that is common to the entire ensemble.
body common control { bundlesequence => { update, main, mypromises("2","3") }; inputs => { "../tests/mod_files.cf", "../tests/mod_files_copy.cf", "../tests/mod_exec.cf", "../tests/mod_process.cf", "../tests/mod_access.cf" }; solaris:: # inputs => { "update.cf", "main.cf" , "linux.cf" }; }
bundle agent main() { files: "/path/file.*" edit_line => myedit("${this}") , access => myaccess, file_select => myfilter, changes => tripwire, recurse => "inf"; "$(filelist)" edit_xml => insertlist("$(filelist)") , edit_line => diddle , access => myaccess , access => others; # ("white"); "/etc/xyz" -> "cf-agent" edit_line => myedit("${this}") , access => myaccess; "/usr/local" linkto => linkdetails("/site/mountpoint/local"); "/var" recurse => "inf", name_select=> "fish.*", tidy => tidymask, rename => rotateme, repository => "/override"; }There are many details here to be expanded upon. For that, we use the body declarations for each promise subject type.
######################################################### body access others(parame) { milkyway:: owner => { "root", "wheel", "sudo" }; } ######################################################### body access myaccess() { any:: mode => "+077,-02"; owner => { "mark","siri" }; solaris:: group => readstringlist("filename"); linux:: group => { "root", "wheel" }; } ######################################################### body linkto linkdetails(tofile) { link_type => "symbolic"; # /absolute/abs/hard/relative/rel copy_patterns => ""; # regex list deadlinks => "kill"; #/force when_no_file => "force"; # kill } ######################################################### body transaction controlbody { loglevel => "usr1"; reportlevel => "inform"; ifelapsed => "10"; expireafter => "20"; } ######################################################### body changes tripwire { hash => "md5"; update => "yes"; } ######################################################### body file_select myffilter # # we can build old "include", "exclude", and "ignore" from these # as standard patterns # { name => { ".*.asc" }; # regex matching file name path => { "/var/.*/mail", "/usr/.*/mail" }; mode => "700"; size => irange("10000,10000000"); owner => { "mark", "cell", "motd" }; group => { "ecg", "mark" }; ctime => irange(ondate(2000,1,1,0,0,0),now()); mtime => irange(ago(1,0,0,2,30,0),now); atime => irange(ondate(1997,2,22,0,0,0),now()); exec_regex => "/usr/bin/file $(this) (.*ascii.*)"; filetypes => { "dir", "link" }; issymlinkto => { "/dev/null", "/dev/tyyS0"}; result => "type&mode"; } ######################################################### body tidy tidymask { age => "0"; size => irange(50000,inf); # number/empty age_type => "mtime"; #ctime/mtime/atime dirlinks => "delete"; #keep/tidy/delete rmdirs => "yes"; #[true/all]/[false/none]/sub links => "stop"; #stop/keep/traverse/tidy } ######################################################### body rename rotateme { newname => "filename"; rotate => "4"; # 0 means empty file size => irange(1,2); action => "nop"; #disable/warn }Promises can also be made about entirely abstract matters that do not require any follow up actions from the strong arm of cfengine. Unrecognized types are simply ignored, so you can use the syntax for your own ends (e.g. note-taking with variable expansion) and perhaps even develop a module to handle such promises as an extension.
bundle external ExoCartography() { # # This means nothing, but I can use it to think out loud... # graph: any:: # classes relate to where these promises are known # not to be confused with the hosts promising # List the various conduits in the network # promise from nexus to cube # Calculate the degree distribution "sphere" -> "cube" label => "+nfs", weight => "0.4"; "sphere" -> "nexus" label => "+nfs", weight => "0.4", # allow all of these simultaneously s_tau => {}, s_chi => {}, # regex i_tau => { "1", "2"}, i_chi => {}, # ranges r_tau => { "0,5"}, # load average r_chi => { "0,2.2" }; # ranges "cube" -> { "nexus", "slogans", "others" } type => "-nfs", to => "nexus"; # default weight = 1.0 ############################################################## sla: # # What can we put in here? # "HIO" -> "opera" tau => "http response time", real_chi => { "0 , 0.0001" }, str_chi => { "^(NaN)" }, availability => "$(www_avail)";
Cfengine 3 has reorganized things to provide fewer but more flexible primitives. What it gives back is a way to recreate all the old explicitness in a user-defined way.
We encourage you to build libraries of templates that you can refer to to make rules more consistent and readable.