From the category archives:

Configuration Management

Encrypting with TrueCrypt

In an earlier post I wrote about some of the free encryption options for PC users.  One of the products I talked about was TrueCrypt, and others agree with me that it is a good option for data encryption.  I decided to take it a step further, and wrote a How-To article using TrueCrypt.  It’s a pretty detailed beginner’s guide to creating encrypted volumes on a Windows PC.  Performing the same operations in Linux wouldn’t be that different, since the interfaces are very similar.

Free Virtualization Options

One of my Suite101 articles covers the Benefits of Virtualization on PCs.  The benefits are summarized here:

  • Run Alternative Operating Systems
  • Safer Internet Browsing and Banking
  • Software Testing on VMs
  • Customized VMs for Multiple Users
  • Snapshots for Easy Backup and Restore

Again deciding that an easy to follow beginner’s guide was warranted, I wrote a Tutorial for getting Linux running in a Virtual Machine on Windows, using VirtualBox.

Encryption and Virtualization for Configuration Management

Both of these technologies can play an important role in config management.  Configuration files containing server information, passwords, etc. are often necessary in the deployment of applications.  Leaving them in plain-text is an invitation for disaster.  Sooner or later, someone who shouldn’t will see the information and gain unauthorized access.  This doesn’t even have to be someone with malicious intent in order to be dangerous.  Some of the worst mis-haps in the tech industry have been caused by well-meaning employees who had more access than their role and expertise warranted.

Virtualization too has it’s advantages for a configuration manager.  If you can create VMs to represent the target servers and workstations, then you can develop, test, and tweak your deployment methods – particularly with respect to automation – without risking any real environment.

These ideas and articles just scratch the surface of the usefulness of encryption and virtualization.  The latter, especially, has a lot to offer in the workplace, including disaster recovery, remote control, backup and restore, etc.   Both technologies can increase the security and reliability of configuration management practices.

{ 0 comments }

I’ve recently received an email saying “Congratulations — you have been chosen by IPED current constituents to become a member of the prestigious Partner Advisory Board exclusively for solution providers. ” It goes on to say that by by participating, I’ll have access to unreleased research information, plus be eligible for other rewards, including gift certificates from Best Buy. To participate, I have to complete surveys on topics like Software Infrastructure, Storage, Security, Systems, etc.

The Advisory Board is hosted by UBM’s Institute for Partner Education & Development (IPED). While I want to feel honored, I keep thinking there’s less to this than meets the eye. A search on the internet only turned up the hosts websites, plus a couple of press releases from companies I’ve never heard of announcing the fact that they’ve been selected to serve on the board.

This is the third or fourth such invitation I’ve received. Maybe this time I’ll follow up on it and see where it leads. If I do, I’ll report my findings here.

This article was originally published by Triond on their web site ComputerSight. I thought it was time to reprint it here, so it appears below in its entirety.


Configuration Management (‘CM’ hereafter) means a lot of different things to different people. Weighty tomes have been written describing the goals, policies, procedures, benefits, pitfalls, and a variety of definitions of CM. One recent CM plan I worked on is a 20-something page document attempting to detail this information and how it relates to the client’s projects.Most of the information available can be boiled down into 4 key concepts, or what can be called the 4 cornerstones of great CM. These concepts represent ideals. The challenge is in the implementation, so that the policies, procedures, and utilities developed support these ideals, or at least the intent behind them.

  1. Version Control : Everything is maintained in a Version Control tool like Serena’s. Some agreed set of items (Configuration Items, or CI’s for short) stored within the tool represent baselines. In other words, they are the set of revisions currently in production. They are not necessarily the most recent revisions.Builds intended for deployment to any post-development environment (QA, Test, Prod, whatever) are always pulled from Version Control, and never copied directly from a development environment.
  2. Separation of Duties and Least Privilege : Actually, these are two principles lumped together because Least Privilege is not possible without Separation of Duties, and Separation of Duties is pointless without Least Privilege. The former simply means that no single person has independent responsibility over more than one area of a system.For example, developers change code, perform unit test, etc., but do not deploy or promote such code to any non-development environment. CM people promote code, but do not develop applications, nor do they approve code changes made by developers (although they may participate in code reviews).
    DBAs have database privileges, but don’t develop application code nor act as system admins. And so on. The Least Privilege principle simply states that no person or running process has more access or system privilege than they need to perform their normal duties or functions at any point in time.Access or privilege for either people or processes can temporarily be increased during the performance of some activity as necessary, then immediately restricted again. Policies implementing these controls make allowances in both these principles for emergency situations.
  3. Auditing : CM personnel periodically conduct audits of applications, systems, and procedures. Any updated application software or configurable item should be traceable to an approved change request, as well as through the entire set of existing quality control, tech review, and change control procedures.This includes not only application executables but database configurations as well. All items are compared with their baseline counterparts in the Version Control repository (ie; the revisions marked as ‘Production’). Discrepancies are reported as non-compliance issues and investigated, and will generally lead to procedural changes designed to eliminate future non-compliance.
  4. Automate, Automate, Automate : This one is an over-riding theme for how we accomplish all this with limited resources. Checking items out of and into Version Control should be quick and painless, and integrated into development IDEs (Interactive Development Environments) if possible. Code promotions are scripted. Database changes are scripted. Auditing utilities are scripted.These scripts themselves are subject to review and kept in version control. Tying it all together gives us reliable, secure systems built with verifiable, repeatable and efficient processes.

{ 0 comments }

As a Configuration Manager, I’m always looking for ways to improve the automation of the builds and deployments of my company’s applications. We use scripts to compile the apps, replace certain token strings with environment-specific values, and copy the new executable code out to the production servers. Ideally, we should not have to use seperate scripts when deploying to different run-time environments (development, integration test, production, etc.). We want instead to pass the target environment into these scripts, and use logic to determine environment-specific values. So I set out to create a Lookup Table to set the values according to the target environment.

I wanted to keep it simple so maintenance would be easy. I wanted it to run in a basic command shell (I use ‘bash’, but most other shells would work as well). UNIX and linux utilities like ‘sed’ and ‘awk,’ and xml parsers would have done the job, but they added complexity so I stayed away from them (although I do use ‘grep’). The listing below is a simplified version of what I came up with. It takes one parameter representing the target environment, and sets 3 variables: the target server, the target database, and a process user ID. It then prints the new values to the screen for verification (an optional step). The script we actually use at work also sets target directories, service names, and website urls, but this is enough to give you the idea:

Listing 1

#!/bin/bash

# Sets environment variables based on lookup string

# Environments: DEV = Development, QA = Quality Assurance,

# UAT = User Acceptance Test, PROD = Production

ENVIRONMENT=$1

# Set server addresses, database names, and user IDs.

line=`grep ^$ENVIRONMENT <<EOF

Env Server Database User ID

— ———————— ——— ———-

DEV dev.myapp.mybusiness.com myappdev devappuser

QA qa.myapp.mybusiness.com myappqa qaappuser

UAT uat.myapp.mybusiness.com myappuat uatappuser

PROD prod.myapp.mybusiness.com myappprod prodappuser

EOF`

set — $line

export AppServer=$2

export DataBase=$3

export UserID=$4

#

# Show environment settings:

echo “AppServer = $AppServer”

echo “DataBase = $DataBase”

echo “UserID = $UserID”

Sample run:

$ ./Lookup.sh DEV

AppServer = dev.myapp.mybusiness.com

DataBase = myappdev

UserID = devappuser

$

Using the Technique

Knowing how this script works is not essential to using the technique, as long as you realize that you can expand it by adding more values to the ends of the input lines, and creating enough values with the ‘export’ statements to accomodate the new values.

>>Read explanation and rest of article >>

{ 0 comments }

Use Secure Shell (SSH) to establish safe, encrypted internet connections through a firewall. With this method, you don’t have to open additional ports through your firewall in order to access external email accounts, access usenet newsgroup servers, and multimedia streams, which leaves your internal network more secure. This means you don’t have to worry about accidentally surfing to a restricted site (which raises red flags in most corporate environments), and can access sites that have been mistakenly blocked by over-zealous monitoring software. Traffic cannot be analyzed for content by sniffers or packet inspection software because of the encryption.

Access to a Server
You will need access to a server running Secure Shell on the other side of the firewall. If you are connecting from inside your company’s firewall, you could run OpenSSH (an open source SSH server) from your home computer or that of a friend. Installing and configuring an SSH server is beyond the scope of this article, but good documentation exists for OpenSSH on its home site. Just make sure that you open a port through any router or personal firewall for SSH traffic. The default is port 22, but you can use any available TCP/IP port. Alternatively, there are some sites that will give you a free shell account on their server running SSH. SilenceIsDefeat.org will give you one for $1.00 if you use paypal (and signing up is then instant), or the cost of a $0.39 stamp if you register through the mail. Finally, if you pay for a commercial web host, many of them allow SSH connections to their servers.

Connection Settings
You also need to know a little about how you connect to the internet from within the firewall. Most companies allow web traffic through a proxy server so they can monitor the content employees are viewing, and can restrict access to sites with objectionable content. You can examine the internet connection settings for your browser. If you are set for “Direct Connection to the Internet” (Firefox), or no proxy or configuration script is set up (IE), then you probably have unfettered (but not necessarily unmonitored) access to the internet, and would only need to use SSH for privacy. You can skip down to SSH Client. If proxy access is set up in a straightforward configuration, then the proxy settings will be displayed right there. Make a note of the address and port of any HTTP or Socks proxies defined. Some companies use an Automatic Proxy Configuration Script, which makes retrieving the proxy settings a little more difficult. If you’ve already noted your proxy settings, you can skip down to SSH Client. Otherwise, here’s some help retrieving your proxy settings from an automatic script. Copy the following lines into a new blank text file and store it on your hard drive:

<HTML>
<HEAD>
<TITLE>Download a file instead of rendering it</TITLE>
<BODY>
<A xhref=”http://URL.OF.SCRIPT” mce_href=”http://URL.OF.SCRIPT” >Right-Click here and select Save As</A>
</BODY>
</HTML>

Edit this file and replace URL.OF.SCRIPT with the address of the automatic configuration script as defined in your connection settings. Save the file as dl.html. . In your browser, use File->Open to open this file. You should see a single link saying to “Right-Click here and Select Save.” Right click on the link select “save link as” or “save target as” (depending on your browser), and save the file locally. You now have a copy of the Automatic Configuration Script which you can peruse in any editor, looking for proxy information. Often a company will use a number of different proxies for different purposes – you’re mainly looking for Socks and HTTP proxies. Note the address and port of any you find. If you think you’ve found an HTTP proxy, you can test it by changing your connection settings and telling your browser to use that server and port explicitly instead of using the Automatic Configuration Script, and connecting to the internet. Just remember to restore the settings afterward.

SSH Client
The last piece you need is a Secure Shell client. This is just a program that opens and manages an encrypted connection to a server. Normally, you would use this combination to log on to, administer, and exchange data between a remote computer (the server) and your local computer (the client). We’re going to be using a more advanced feature of SSH known as “port forwarding”, which lets you direct other network traffic through such a connection. A good choice for Windows users is PuTTY, which can be downloaded freely from the Download Site. Macs and linux users will almost certainly have a good SSH client installed. I’ll be using PuTTY in the examples in the remainder of this tutorial, but the principles will be the same in any SSH client.


PuTTY Configuration
The PuTTY documentation does a good job covering configuration, so I’m just going to focus on the essentials for port forwarding. Basically, you enter the IP address or DNS name of the host to which you’ll be connecting, the port it uses, and a descriptive name in the ‘Saved Sessions’ field. If your browser uses a proxy server to access the internet, then you will configure PuTTY to use the same one(s). In PuTTY’s ‘Category’ tree (left portion of window), click the ‘+’ sign next to ‘Connection’ and click on ‘Proxy.’ If the browser had a Socks proxy configured, select that type in PuTTY. If not, but it had an HTTP proxy configured, then select that type. Enter the Proxy hostname and port that you previously noted. When this is done (or if you didn’t have to add proxy configuration), click back up on the ‘Session’ category and click the [Save] button. Then click the [Open] button. If everything is correct, you should get a new window with a login prompt from the remote system. You’re ready for the last step – actually forwarding a port or two.

There are two methods of forwarding ports through an SSH connection. “Dynamic” forwarding is easier to configure and more flexbile, but can only be used by applications that support a Socks proxy. This includes most modern web browsers, so we’ll start with setting up secure, encrypted web browsing. If you actually logged in, type ‘exit’ and press the key. Otherwise, just close the window. Bring up Putty again, select the ‘Saved Session’ you stored earlier and click the [Load] button. In the ‘Category’ tree, expand ‘Connection’, ‘SSH’, and select ‘Tunnels.’ In the ‘Source port’ field, enter 8081. This can actually be any number higher than 1024, I’m just using 8081 as an example. Select the ‘Dynamic’ radio button, then click the [Add] button. ‘D8081′ will appear in the ‘Forwarded ports’ field. Under Categories, select ‘Session’ (you may have to scroll up to see it), and click the [Save] button again. Click [Open] and log in to the remote server. In your browser, you’ll need to change your proxy settings. Make sure to write down the current settings, so you can restore them later. You are going to set the browser to use a Socks proxy (which dynamic forwarding creates for you). In IE under Lan connection settings you have to select the [Advanced] button to see the proxy configuration fields. In Firefox, they’re visible in the Network ‘Settings’ panel. Add the word localhost as the proxy host or proxy server address, and 8081 as the port. Click [OK] until you are out of the configuration screens. If you can now browse the web, then congratulations, you’re doing so in a secure, encrypted tunnel. Nobody can see what sites you visit unless they watch over your shoulder. Note: examination of your computer’s cache, log files, history, and other forensic evidence will still yield information on your surfing habits. A secure tunnel only protects the data in transit.

The second method of forwarding ports involves forwarding each port used by your network application from your local PC to the actual server running the network service you wish to access. You then configure the application to use your local machine as the server. For example, to connect to your external pop3 mail server, in PuTTY you would go back to the ‘Tunnels’ configuration screen. Add 1110 as the Source port, select the ‘Local’ radio button, enter your mail server’s address followed by ‘:110′ in the ‘Destination’ field, and click [Add]. You should see something like L1110 your.mail.server:110 appear in the ‘Forwarded Ports’ field. Once again, return to the ‘Sessions’ screen and click [Save]. Fail to do this after any changes, and you’ll lose them. Now, open your mail client. Wherever you would normally enter your pop3 server address and the port it uses, enter ‘localhost’ and 1110. In Outlook Express for example, you will find these settings in the ‘Servers’ tab and the ‘Advanced’ tab in the Account Properties screen. Once you’ve made these changes, you should be able to connect to the SSH server using your saved session in PuTTY, then retrieve and read your mail in your mail client software. Sending mail uses a different port (25),and often a different server name, so you’ll have to forward another port in a similar manner if you want to be able to send mail as well.

Pretty much any network service that uses a defined port or set of ports can be configured to work through an SSH tunnel in this manner. This includes services that your company may ordinarily block, like Instant Messaging services, Usenet Newsgroup access, streaming music sites, etc. Note that anyone with access to network sniffers or inspection software, be they crackers, hackers, or network admins, will still be able to see network traffic between your computer and the remote SSH server, they just won’t be able to tell what it is or where it goes beyond that point.

{ 3 comments }

More Secure Shell troubles

by joe on December 5, 2007

Well, I still haven’t solved the earlier problems (see other posts in this category), but now I’m having a new problem. One of the Windows servers we’ve had OpenSSH running on for quite some time suddenly seems have issues. It will stop accepting connections. The message in the sshd.log is always some variation of this:

63 [main] sshd 7632 child_copy: linked dll bss write copy failed, 0x207A000..0x207CAA0, done 0, windows pid 8136, Win32 error 998

Stopping the service, with the intent of restarting, didn’t work, as the service would then not start at all. cygrunsrv -S sshd would yield the mysterious win32 error 1062, and would refuse to start, with nothing showing up in the event logs. A complete re-installation of cygwin fixed the problem, but it returned within one day. Now I find out that this server is short on memory (it’s used for some heavy-duty data processing), so I suspect that the problem is related to that. If you’re researching the same issue, check your available memory. I’ll report more details here as they develop. In the end, I’ll probably write a comprehensive article for publication on Associated Content.

Update: 12/06/2007: Some of our scripting relied on multiple successive ssh connections to a target server. The idea was to maintain as much of the scripting logic as possible on our build server, executing remote commands one at a time, each via an SSH connection. This may have caused a resource bottleneck. I re-wrote some of the scripts to do a number of things in a single connection. I also added retry logic, in case of the “resource unavailable” error. We’ll see how it goes.

{ 0 comments }

I recently ran a training session teaching some Configuration Management (CM) personnel some of the basic UNIX/linux shell commands, along with some of the common ways Secure Shell (SSH) utilities can be used to move application code around during deployments. I created an outline for the class, which is reproduced below. In no way is this outline a complete reference for using shell commands and SSH for CM, but it introduces some of the basic utilities and commands that can be a part of a comprehensive CM architecture.

I) bash : a *nix shell
A) A shell is a command-line interface to an OS. There are lots of shells available in *nix (korne, bourne, etc.). bash tries to include the best features of each. Shells are related to DOS.
B) cygwin makes it work in Windows, along with most other “POSIX” compliant programs & utilities (including OpenSSH).
C) Some common shell commands (all of these work in the other shells as well):

i) cd : Change working directory.

(a) cd : By itself, cd puts you in your own home directory.
(b) cd /tmp/ftp_files : puts you in the /tmp/ftp_files directory. The leading “/” means start at the root or base of the file system, and traverse from there.
(c) cd myfiles : puts you in a subdirectory from your current location called myfiles. You could be anywhere in the file system and this form of the command will only look there for the named sub-directory.

ii) ls: List files.

(a) ls : lists files in current directory
(b) ls /usr/bin : lists files in sub-directory /usr/bin
(c) ls -l : lists files in “long” format, showing owner, permissions, sizes, etc.
(d) ls BAM* : Lists all files in current directory whose names start with “BAM”. The “*” is a wild-card.

iii) cp : Copies files from place to place, optionally with new name.

(a) cp thisfile.txt thatfile.txt : makes copy of thisfile.txt with name thatfile.txt.
(b) cp /tmp/sales.wks /home/jp : Copies file named sales.wks from directory /tmp to directory /home/jp (assuming this directory exists).
(c) cp /var/news/daily/* ~ : Copies all files (using a wild-card again) from directory /var/news/daily to the user’s home directory. The “~” by itself means current user’s home.

iv) grep : Matches a string with some source of text, often the contents of a file.

(a) grep error: *.log : Searches all files in the current directory whose names end in .log for any lines containing the text “error:”. If it finds any, it lists the file name along with the actual line of matching text.
(b) grep -i virus ~brian/* : Searches all the files in user Brian’s home directory for any file which includes the term “virus”. The “-i” switch makes the search case-insensitive. The “~” followed immediately by a user name is short-hand meaning the named user’s home directory.

[click to continue...]

{ 3 comments }

The “Could not create an instance of the CmdLib object. Please register the Microsoft.CmdLib component.” error message was because of certain web server extensions that weren’t installed. I think the SQL Server Reporting Server needed to be installed. In any case, we’re past that now.

Current problem: we’re running the sshd service with a Domain Admin ID. This works, but poses a security risk. I’m trying to get a test installation working where the service ID is a Domain User but not a Domain Admin. The service starts, but anyone connecting to it is dis-connected immediately after authentication. The debug message (running the server with full debug messages logged) states “fatal: setreuid 14153: No such process.” So again, looking for any pointers. I’ve been all over Google about this, nothing that seems to apply yet.

{ 0 comments }

Recently it was decided at the large State government facility where I work that Secure Shell (SSH) would be used to facilitate the deployment of application software to the servers. This approach has a number of advantages. All data transfers are encrypted. Key pairs can be used to automate the authentication, so the entire transfer can be scripted with Shell scripts. Plus, with SSH’s ability to execute remote commands, command-line utilities on the target server could be utilized to stop and start services and web sites as necessary during the deployments.

Since most of the target servers that host our applications are Windows servers, that meant installing a 3rd party SSH server. The State opted for OpenSSH, the Open Source SSH implementation. To get that to work in a Windows environment requires Cygwin, a “linux-like” shell environment that runs under Windows. Getting this to work in a default environment is a snap. Getting it to work in a complex environment which includes Active Directory, domain controllers, and Group Policy Objects has proved to be quite a challenge. We are making progress, and I’ll probably write up the entire process in a “How-To” article in the near future. In the mean-time, I’m struggling with one weird error trying to stop and start web-sites.

We are using a command-line utility named iisweb.vbs to stop and start the services. The ID we are running the script under appears to have the necessary privileges to use this utility (being a Domain Admin), but when it executes, it errors out with the message “Could not create an instance of the CmdLib object. Please register the Microsoft.CmdLib component.” The funny thing is, this was working until yesterday, when the server teams re-built the server in order to re-partition the drives. Any help would definitely be appreciated.

Jp

Next post in series…

{ 2 comments }

Secure Shell Quickie

by joe on October 15, 2007

While waiting for Associated Content to publish the Charity Music article (something they’ve now agreed to do), I wrote a quick little overview of SSH and published it on Helium, where it’s now rated #1 of 2. What is SSH? The nice thing about Helium is that your submissions are published more or less immediately.

{ 0 comments }

Recent Articles published elsewhere

by joe on September 18, 2007

Configuration Management: 4 Key Concepts

Choosing a Construction Contractor

{ 0 comments }