Wednesday, February 28, 2007

JET: Controlling custom_files with a custom extension

Any site running Sun hardware with more than one system should be looking at JumpStart to ensure that systems can be rebuilt consistently. the corollary to this is that any site running JumpStart environment should be using Sun's Jumpstart Enterprise Toolkit (JET). JET provides a consistent framemwork for accomplishing most common tasks, and a consistent framework to write extensions within. Standards and discipline are good.

One of the modules which comes with JET is called simly enough, custom. The custom module allows you specify either packages or files which should be added to a server during any of N predetermined reboots. This allows you to ensure that a change which requires a reboot can be made prior to a dependent process being started. Sounds good so far.

Following a recent Solaris 9 server build I was perusing the system for problems by auditing log files. In the messages file I discovered some lines indicating that a Kerberos problem was rearing its ugly head:
Kerberos mechanism library initialization error: No profile file open.
Our site does not use Kerberos, so it had to be a recent configuration change - not surprising considering we had just updated the patch set. After some research I arrived at BugID 5020096. This bug indicates that the issue can be resolved by removing some offending lines from /etc/krb5/krb5.conf.

This should be easy enough to fix in future builds. Just add the modified krb5.conf to the JET template's custom_files variable, and we'll be in good shape. Ahh, not so fast. How will we know what the file originally contained? A true Solaris Jedi will always manage an audit trail of his activities. If I were making the change manually I would copy the file to file.orig, or file.datestamp. Automation is not an excuse for abandoning discipline.

The trouble with JET is that its custom module's functionality for installing files is limited to two operations: overwrite or append. Overwrite simply clobbers any file which may exist. For example, to install the /etc/motd file I would palce my custom file in the configured JET file location, then add a line like this to the JET template:
custom_files_1="motd:o:/etc/motd"

motd is a fairly harmless little file, but knowing little about Kerberos, I dind't want to blindly whack the original file. The right solution to this problem lies in creating a simple extension to the JET toolkit. I began by examining the code from the custom module. Two modules specifically are relevant to this project: install, and postinstall. Within them is a simple case statement which handles the "o" or "a" functionality:

case ${mode} in
a) case ${fn2} in
/etc/hosts) JS_merge_hosts ${filefound};;
*) JS_cat ${filefound} ${ROOTDIR}${fn2}
;;
esac;;
o) JS_cp ${filefound} ${ROOTDIR}${fn2};;
esac


So, when I use an "o" in my custom_files module, it called JS_cp. I now needed to find the library which contains these core functions. Eventually, a colleague and I traced it back to /opt/SUNWjet/utils/lib. Looking at the JS_cp function revealed exactly what I expected: a simple copy routine wrapped in some voodoo.

Feeling a bt optimistic, I copied JS_cp to JS_cp_preserve and modified the code a bit so it would first check to see if the destination file exists, and if so, backup the file with a datestamp. Once the backup was in place, the original copy operation was performed. This was very trivial shell scripting. Here's what I ended up with:

if [ "$#" != "2" ]; then
JS_error "`basename $0`: Illegal Arguments. Usage: "
fi

JS_FROM=$1
JS_TO=$2

JS_display "Copying file `echo ${JS_FROM} | sed -e \"s?^${SI_CONFIG_DIR}/??\"` to ${JS_TO}"

if [ -f ${JS_TO} ] ; then
datestamp="`/usr/bin/date +%Y%m%d`"
/bin/cp -p ${JS_TO} ${JS_TO}.jet.${datestamp}
case $? in
0) # Success
JS_display "Successfully preserved ${JS_TO}.jet.${datestamp}"
;;
1) # Failure
JS_display "WARNING: Failed to preserve original file ${JS_TO}"
;;
esac
fi

/bin/cp -p ${JS_FROM} ${JS_TO}

if [ "$?" != "0" ]; then
JS_error "JS_cp:\t\tError occured while copying ${JS_FROM} to ${JS_TO}"
fi


Next, I returned to the install and postinstall code, and modified the case statements to accept a "b" operation (b for backup). I then executed a test Jump and was very pleased to see my JET extension had worked! I can now have custom_files install the workaround krb5.conf, and maintain a backup of the original. Here's the modified code:

case ${mode} in
a) case ${fn2} in
/etc/hosts) JS_merge_hosts ${filefound};;
*) JS_cat ${filefound} ${ROOTDIR}${fn2};;
esac;;
o) JS_cp ${filefound} ${ROOTDIR}${fn2}
b) JS_cp_preserve ${filefound} ${ROOTDIR}${fn2};;
esac


Note that you need to make this modification in both /opt/SUNWjet/Products/custom/isntall and postinstall.

Now, all I need to do it specify something in the custom_files module like this:
custom_files_N="krb5.workaround:b:/etc/krb5/krb5.conf"

And I will get a clean backup of the original file. Such a simple tweak - I hope the Sun folks who maintain JET will add something similar. While some limitations of JET can be frustrating, its intuitive layout and ease of extension make it something I grow more fond of each time I use it.

4 comments:

mike said...

I stumbled upon your article. Looks like a nice RFE for JET. I'll sneak it into the next release. (4.4.7)

Christopher Hubbell said...

That would be fantastic - thanks for listening!

Chris Peck said...
This comment has been removed by the author.
Chris Peck said...

I just updated to JET 4.4.7 and was very happy to see this added to it. Nice addition (better than the hack I had in place for it)...