Shell scripts are often a quick, dirty way to get the job done. You glue together external tools, maybe do a little error checking and process all data as strings.
This is great for some very simple problems but as requirements change and more is demanded from the code shell scripts become unwieldy and fragile. When they get large, they become slow and difficult to maintain. If you need to write robust code then shell is not the way to go.
At the other extreme we have Haskell. Haskell is about as far from shell programming as you can get: its full of abstractions, its designed for robust error and exception handling, is strongly statically typed (you’d be shot if you represented all data as strings). Fortunately, it is also rather concise, like shell code.
So it makes sense then for Haskell to be used in a number of “scripting” situations where robustness and correctness are important. For example, large, critical tools, such as the package management infrastructure in the Linspire linux distro, are written in Haskell.
This article looks at how to use Haskell for scripting tasks. By refining the semantics of the problem domain, employing abstract, we produce shorter and more robust code. Finally, as a highlight, we’ll use type checking to statically separate code that requires root privileges from user code.
The spec
I have a variable frequency cpu in my laptop. When the frequency is low the battery life is greatly extended, and the machine stays a lot cooler. At the highest level, my code runs a faster.
There exist tools for all common operating systems to automatically scale up and down the clock based on load. However, I usually don’t care about scaling — I either explicitly want the clock all the way up, or all the way down. In particular, when I do benchmarking I want to keep the cpu clocked up all the way.
So we’ll develop a simple program that acts as a toggle, flipping the cpu speed up or down, and printing some strings about the current state. It should behave like this:
$ cpuperf
cpu: 0 -> 100
clock: 1.6 Ghz
$ cpuperf
cpu: 100 -> 0
clock: 0.6 Ghz
Operating details
First let’s look at how we’d typically do this in the shell.
I use the OpenBSD operating system. Rather than using a /proc filesystem as on linux, tuning kernel variables in OpenBSD is done via sysctls. The userland sysctl program lets you get or set kernel values:
For example, the OS type:
$ sysctl kern.ostype
kern.ostype=OpenBSD
The current clock speed:
$ sysctl hw.cpuspeed
hw.cpuspeed=600
The current performance level (between 0 and 100):
$ sysctl hw.setperf
hw.setperf=0
We’ll use these latter two sysctls to tweak the clock speed. Note that to set a sysctl value we need root privileges (via sudo).
An implementation in shell
Implementing the specification in shell:
#!/bin/sh s=`sysctl hw.setperf` old=`echo $s | sed 's/.*=//'` if [ "100" = $old ] ; then new=0 else new=100 fi sudo sysctl -w hw.setperf=$new > /dev/null printf "cpu: %d -> %dn" $old $new speed=`sysctl hw.cpuspeed` clock=`echo $speed | sed 's/.*=//'` clock=`bc -l -e "$clock / 1000" -e quit` printf "clock: %0.1f Ghzn" $clock
Note that we assume you’ve made the sysctl command accessible through sudo. For example:
$ visudo
...
dons mymachine = NOPASSWD: /sbin/sysctl -w hw.setperf=0
dons mymachine = NOPASSWD: /sbin/sysctl -w hw.setperf=100
...
The script is short and does no error handling. Does it work?
$ sh naive.sh
cpu: 0 -> 100
clock: 1.6 Ghz
$ sh naive.sh
cpu: 100 -> 0
clock: 0.6 Ghz
$ sh naive.sh
cpu: 0 -> 100
clock: 1.6 Ghz
Great! The performance is toggled between 0 and 100, clocking up and down the cpu. Some interesting things to note;
- we use regular expressions for parsing
- we don’t check for failure
- strings are treated as numbers
- floating point math is a little hard
- we take root privileges in the middle of the code
A Haskell translation
We can directly translate this code into Haskell:
import Text.Printf import Process main :: IO () main = do s <- run "sysctl hw.setperf" let old = clean s new = if old == 100 then 0 else 100 :: Integer run $ "sudo sysctl -w hw.setperf=" ++ show new printf "cpu: %d -> %dn" old new s <- run "sysctl hw.cpuspeed" let clock = fromIntegral (clean s) / 1000 printf "clock: %f Ghzn" (clock :: Double) where clean :: String -> Integer clean = read . init . tail . dropWhile (/='=')
We replace the regular expression with some list processing, failure is translated to unhandled exceptions, IO is interleaved with pure actions (like the math), just as in shell. One difference is that we explicitly treat strings as Integers and Doubles.
Running the code in the bytecode interpreter:
$ runhaskell naive.hs
cpu: 100 -> 0
clock: 0.6 Ghz
$ runhaskell naive.hs
cpu: 0 -> 100
clock: 1.6 Ghz
Of course, this being Haskell, we can compile to native code:
$ ghc -O --make naive.hs -o cpuperf
[1 of 2] Compiling Process ( Process.hs, Process.o )
[2 of 2] Compiling Main ( naive.hs, naive.o )
Linking cpuperf ...
$ ./cpuperf
cpu: 100 -> 0
clock: 0.6 Ghz
Which does run quite a bit faster than bytecode (and faster than the sh code). This code uses the Process module, a small wrapper over System.Process.
Doing a better job
This is all very nice, but the code feels a bit icky. There’s something unsatisfying: we haven’t really captured the sysctl abstraction at all, so there’s no easy reuse of this code for other purposes. Neither have we looked at error handling, and finally, we’ve played fast and loose with sudo. In a larger application, we’d want to be far more careful about taking root privileges.
Domain specific shell code
The first thing to clean this code up is to notice that the sysctl values behave like mutable boxes (these are known as ‘variables’ in some cultures). A nice interface to mutable boxes is the get/set/modify api, which goes something like this:
get :: box -> m a set :: box -> a -> m () modify :: box -> (a -> a) -> m (a,a)
The ‘get’ function retrieves a value from a mutable box. The set function writes a new value into one. The most convenient function is `modify’, a higher order function which takes a box, and a function modifying the contents, and applies that to the current contents, mutating the contents. It returns the old and new values of the box.
Since sysctls act as mutable boxes of integers keyed by strings names our abstract api can be specified concretely as:
get :: String -> IO Integer set :: String -> Integer -> Priv () modify :: String -> (Integer -> Integer) -> IO (Integer, Integer)
We can implement the semantics of the ‘sysctl’ command as a small domain specific set of functions in Haskell:
get s = do v <- run ("sysctl " ++ s) readM (parse v) where parse = tail . dropWhile (/= '=') . init set s v = run $ printf "sysctl -w %s=%s" s (show v)
and our nice ‘modify’ function combines the two:
modify s f = do v <- get s let u = f v set s u return (v,u)
This lets us simplify the main function:
main = do (old,new) <- modify "hw.setperf" toggle clock <- get "hw.cpuspeed" printf "cpu: %d -> %dn" old new printf "clock: %f Ghzn" (fromIntegral clock / 1000 :: Double) toggle v = if v == 100 then 0 else 100
Which is really pretty nice. By getting closer to the semantics of the problem, we find the right api, and the code becomes simpler and cleaner. So our code now more closely matches the spec of:
- modify the hw.setperf value based on its current value
- print the current cpu speed
Improving error handling
In the current code exceptions aren’t caught (if they’re noticed at all). We can introduce a bug to see the problem:
parse = read -- . init . tail . dropWhile (/='=')
Now the Haskell code dies with the unhelpful error message:
$ cpuperf
*** Exception: user error (Prelude.read: no parse)
We really should handle the possibility of ‘read’ failing. Currently, any error results in a call to the default ioError action in the IO monad.
However, this being Haskell, we can implement our own error monad to provide custom error handling. This situation is exactly what the ErrorT monad transformer. was designed for. So how to use it?
The first step is to replace read with a version lifted into a generic error monad, MonadError:
readM :: (MonadError String m, Read a) => String -> m a readM s | [x] <- parse = return x | otherwise = throwError $ "Failed parse: " ++ show s where parse = [x | (x,t) <- reads s]
Now should a parse fail it will call the ‘throwError’ function in whatever monad we happen to be using — the code is polymorphic in its monad type. For particular types, we can see how throwError is defined:
instance MonadError IOError IO where throwError = ioError instance (Error e) => MonadError e (Either e) where throwError = Left
That is, for IO, throwError corresponds to a normal io error (which will throw an exception). If we’re in the Either monad, instead our result will be marked as an error (with no exception thrown).
But, even with this nice ‘read’ function, we still have a problem checking errors. Functions like ‘get’ or ‘set’ might fail. One way to handle errors like this is to check every functions’ result (this style is encouraged in some cultures). We can tag any error and then check the result after each function call using the Either type:
data Either a b = Left a | Right b
A value of ‘Right x’ is a good value, anything of the form ‘Left e’ is an error. Assuming we then wrap ‘get’ and ‘set’ to return ‘Left’s in the case of errors, we can obfuscate our ‘modify’ function with error handling boilerplate like so:
modify :: String -> (Integer -> Integer) -> IO (Either String (Integer,Integer)) modify s f = do ev <- get s case ev of Left e -> return (Left e) Right v -> do let u = f v ev <- set s u case ev of Left e -> return (Left e) Right _ -> return (v,u)
Urgh .. boilerplate! Note the common pattern: after each evaluation step: we perform a particular check, and then optionally propagate results further down.
All good Haskellers reading should immediately recognise the pattern:
- we have a particular operation we need to run between each step of our code
This kind of boilerplate can be abstracted perfectly with a monad (of course).
Scrap your error handling boilerplate
But which monad? Well, Either is itself an monad: the Error monad:
instance (Error e) => Monad (Either e) where return = Right Left l >>= _ = Left l Right r >>= k = k r
If you recall from the dozens of other monad tutorials out there, a monad gives us a programmable ‘;’ (the semicolon statement terminator from the imperative world). With a custom monad we can specify precisely what happens at the end of each statement in our code.
in this case, we want any ‘Left’ value to immediately terminate the computation, and any ‘Right’ value to produce a result we feed to the rest of the code. Since we need to use IO as well, we’ll actually need an ErrorT monad transformer, which wraps an underlying monad with error handling capabilities:
newtype ErrorT e m a = ErrorT { runErrorT :: m (Either e a) }
Note that body of ‘ErrorT’ is exactly the type of our explicit boilerplate full code:
IO (Either String (Integer,Integer))
where
m = IO e = String a = (Integer,Integer)
We can thus scrap our boilerplate, and rewrite modify to run in a new ErrorT monad. We replace the use of IO and Either with a new monad, Shell, with its own MonadError instance:
newtype Shell a = Shell { runShell :: ErrorT String IO a } deriving (Functor, Monad, MonadIO)
In this way any errors thrown will be translated to useful strings in the Shell monad. We can now implement a custom ‘throwError’ for our Shell monad:
instance MonadError String Shell where throwError = error . ("Shell failed: "++)
running a fragment of Shell code is achieved with:
shell :: Shell a -> IO (Either String a) shell = runErrorT . runShell
And our ‘modify’ function has its boilerplate entirely moved into the ‘;’ :
modify :: String -> (Integer -> Integer) -> Shell (Integer, Integer) modify s f = do { v <- get s; let u = f v; set s u; return (v,u); }
Of course, since this is Haskell, we can scrap our (programmable) semicolons too, and just specify which ‘;’ to use in the type:
modify :: String -> (Integer -> Integer) -> Shell (Integer, Integer) modify s f = do v <- get s let u = f v set s u return (v,u)
Finally, running this code, we get the much nicer, and more specific, error output:
cpuperf: Shell failed: Failed parse: "hw.setperf=0n"
The error handling boilerplate is hidden by the error handling monad, inside the invisible, programmable ‘;’.
Adding privilege separation
One slightly icky thing at the moment is the use of sudo directly in the code to obtain root privileges. In larger software the use and abuse of root privileges can be a source of security problems. Some projects go to great length to precisely control the scope of code that has root privileges using privilege separation.
This kind of property is the kind of thing we can lean on the type system for: to implement statically checked privilege separation.
To do this we need to introduce a new type for actions that run with root privileges:
newtype Priv a = Priv { priv :: Shell a } deriving (Functor, Monad, MonadIO)
Yes! Another monad! It’s really just the Shell monad dressed as a new type, so we can distinguish the two in the type checker. Note how we lean heavily on GHC’s newtype deriving to automatically generate boilerplate code implementing the basic type classes for our type.
Now we add a custom error message for any code that fails in privileged mode:
instance MonadError String Priv where throwError = error . ("Priv failed: "++)
The key step is to abstract out the taking of root ops into a combinator, and then hiding the Priv constructor:
runPriv :: String -> Priv String runPriv = Priv . run . ("/usr/bin/sudo " ++)
Now the only way to get Priv status in your types is to actually run the code through ‘sudo’. So the type ‘Priv’ means ‘this code will be checked by sudo’.
Our set sysctl code becomes:
set :: String -> Integer -> Priv String set s v = runPriv $ printf "sysctl -w %s=%s" s (show v)
and we explicitly state in the type of ‘set’ that it runs in the Priv monad, not the normal Shell monad.
The cool thing is that we can ask the typechecker now to audit our code for all uses of priv commands that are unchecked. Compiling the old code, we get:
Main.hs:66:4: Couldn't match expected type `Shell t' against inferred type `Priv String'
Great! On line 66 we use a program requiring root privileges as if it was a normal user command, the ‘set’ call in ‘modify’. So now we can check that that is indeed a place we should be taking root ops, and then tag it as safe with ‘priv’:
modify :: String -> (Integer -> Integer) -> Shell (Integer, Integer) modify s f = do v <- get s let u = f v priv (set s u) return (v,u)
which evaluates runs a fragment of Shell code in the Priv monad. So, if in doubt, embed the problem domain in the type system.
Summary
The final code, with error handling and privilege separation on the type level boils down to:
import Shell import Text.Printf main = shell $ do (old,new) <- modify "hw.setperf" toggle clock <- get "hw.cpuspeed" io $ do printf "cpu: %d -> %dn" old new printf "clock: %f Ghzn" (fromIntegral clock / 1000 :: Double) toggle v = if v == 100 then 0 else 100
All the rest is library code. For binding to ‘sysctl’ nicely:
-- -- Read a sysctl value from the shell -- get :: String -> Shell Integer get s = readM . parse =<< run ("sysctl " ++ s) where parse = tail . dropWhile (/= '=') . init -- -- Set a sysctl value. Runs in the Priv monad, and requires root privledges. -- Will prompt for a password. -- set :: String -> Integer -> Priv () set s v = do runPriv $ printf "sysctl -w %s=%s" s (show v) return () -- -- Modify a particular sysctl value, using a function applied to the -- current value, yielding a new value. Both the old and new values are -- returned. -- modify :: String -> (Integer -> Integer) -> Shell (Integer, Integer) modify s f = do v <- get s let u = f v priv (set s u) -- root return (v,u)
And the Shell and Priv monads are implemented as:
{-# OPTIONS -fglasgow-exts #-} -- -- Provides a Shell and Priv monad, for encapulating errors in -- shell programs nicely, and for static separation of code requiring root -- privledges from other code. module Shell where import qualified Process import System.IO import System.Exit import Text.Printf import Control.Monad.Error import Control.Exception -- -- The 'Shell' monad, a wrapper over IO that captures failure in an -- error transformer. -- newtype Shell a = Shell { runShell :: ErrorT String IO a } deriving (Functor, Monad, MonadIO) -- -- The 'Priv' monad, a shell monad for commands requiring root -- privledges. Let's us distinguish such command statically, on the type -- level. -- -- To run something in the Priv monad, use 'priv'. -- newtype Priv a = Priv { priv :: Shell a } deriving (Functor, Monad, MonadIO) -- -- Rather than just derive error handling, we'll roll our own that -- propagates shell failures into errors. -- instance MonadError String Shell where throwError = error . ("Shell failed: "++) instance MonadError String Priv where throwError = error . ("Priv failed: "++) -- Run a normal shell command as the user. Return either a result or an error value shell :: Shell a -> IO (Either String a) shell = runErrorT . runShell -- Run a privileged command, requiring sudo access. Return any output runPriv :: String -> Priv String runPriv = Priv . run . ("/usr/bin/sudo " ++) -- -- Convenient wrapper -- io :: IO a -> Shell a io = liftIO -- -- Run a shell command, wrapping any errors in ErrorT -- run :: String -> Shell String run = io . Process.run
The entire program is packaged up by Cabal, and available online from Hackage, the central repository of new haskell code and libraries.
Running the damn thing:
$ cpuperf
cpu: 100 -> 0
clock: 0.6 Ghz
$ cpuperf
cpu: 0 -> 100
clock: 1.6 Ghz
$ cpuperf
cpu: 100 -> 0
clock: 0.6 Ghz
$ cpuperf
cpu: 0 -> 100
clock: 1.6 Ghz
The final act is to bind the Haskell program to my ThinkPad’s “Access IBM” hotkey:
tpb -d -t /home/dons/bin/cpuperf
So hitting ‘Access IBM’ now runs the cpu clock scaling Haskell program.
Control.Monad.Writer 