<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>thinking sysadmin &#187; freebsd</title>
	<atom:link href="http://andyleonard.com/category/freebsd/feed/" rel="self" type="application/rss+xml" />
	<link>http://andyleonard.com</link>
	<description>qstat -u aleonard -s z</description>
	<lastBuildDate>Tue, 28 Feb 2012 04:47:09 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.2</generator>
		<item>
		<title>Automatic ZFS Snapshot Rotation on FreeBSD</title>
		<link>http://andyleonard.com/2010/04/07/automatic-zfs-snapshot-rotation-on-freebsd/</link>
		<comments>http://andyleonard.com/2010/04/07/automatic-zfs-snapshot-rotation-on-freebsd/#comments</comments>
		<pubDate>Thu, 08 Apr 2010 03:59:02 +0000</pubDate>
		<dc:creator>Andy</dc:creator>
				<category><![CDATA[freebsd]]></category>
		<category><![CDATA[anacron]]></category>
		<category><![CDATA[auto-snapshot]]></category>
		<category><![CDATA[bash]]></category>
		<category><![CDATA[snapshot]]></category>
		<category><![CDATA[zfs]]></category>

		<guid isPermaLink="false">http://andyleonard.com/?p=460</guid>
		<description><![CDATA[OpenSolaris has ZFS Automatic Snapshots; FreeBSD, while it has ZFS, doesn&#8217;t have a comparable feature that I&#8217;m aware of. So I wrote my own, zfs-snapshot.sh: From the command line, call the script something like the following: This would take a recursive snapshot of the &#8220;tank&#8221; zpool with the basename weekly, rotating through five snapshots with [...]]]></description>
			<content:encoded><![CDATA[<p>OpenSolaris has <a href="http://blogs.sun.com/timf/en_IE/entry/zfs_automatic_snapshots_in_nv">ZFS Automatic Snapshots</a>; FreeBSD, while it has ZFS, doesn&#8217;t have a comparable feature that I&#8217;m aware of.  So I wrote my own, <code>zfs-snapshot.sh</code>:<br />
<span id="more-460"></span></p>
<pre class="brush: bash; title: ; notranslate">
#!/usr/local/bin/bash

# Path to ZFS executable:
ZFS=/sbin/zfs

# Parse arguments:
TARGET=$1
SNAP=$2
COUNT=$3

# Function to display usage:
usage() {
    scriptname=`/usr/bin/basename $0`
    echo &quot;$scriptname: Take and rotate snapshots on a ZFS file system&quot;
    echo
    echo &quot;  Usage:&quot;
    echo &quot;  $scriptname target snap_name count&quot;
    echo
    echo &quot;  target:    ZFS file system to act on&quot;
    echo &quot;  snap_name: Base name for snapshots, to be followed by a '.' and&quot;
    echo &quot;             an integer indicating relative age of the snapshot&quot;
    echo &quot;  count:     Number of snapshots in the snap_name.number format to&quot;
    echo &quot;             keep at one time.  Newest snapshot ends in '.0'.&quot;
    echo
    exit
}

# Basic argument checks:
if [ -z $COUNT ] ; then
    usage
fi

if [ ! -z $4 ] ; then
    usage
fi

# Snapshots are number starting at 0; $max_snap is the highest numbered
# snapshot that will be kept.
max_snap=$(($COUNT -1))

# Clean up oldest snapshot:
if [ -d /${TARGET}/.zfs/snapshot/${SNAP}.${max_snap} ] ; then
    $ZFS destroy -r ${TARGET}@${SNAP}.${max_snap}
fi

# Rename existing snapshots:
dest=$max_snap
while [ $dest -gt 0 ] ; do
    src=$(($dest - 1))
    if [ -d /${TARGET}/.zfs/snapshot/${SNAP}.${src} ] ; then
	$ZFS rename -r ${TARGET}@${SNAP}.${src} ${TARGET}@${SNAP}.${dest}
    fi
    dest=$(($dest - 1))
done

# Create new snapshot:
$ZFS snapshot -r ${TARGET}@${SNAP}.0
</pre>
<p>From the command line, call the script something like the following:</p>
<pre class="brush: bash; light: true; title: ; notranslate">
./zfs-snapshot.sh tank weekly 5
</pre>
<p>This would take a recursive snapshot of the &#8220;tank&#8221; zpool with the basename weekly, rotating through five snapshots with names &#8220;weekly.0&#8243; through &#8220;weekly.4&#8243;.  This allows you to implement a snapshot scheme approximately similar to NetApp&#8217;s hourly-daily-weekly scheme, if you like.  Because my FreeBSD workstation isn&#8217;t on 24&#215;7, I run hourly snapshots out of <code>/etc/crontab</code>:</p>
<pre class="brush: bash; gutter: false; title: ; notranslate">
# Automated ZFS backups (hourly):
0 * * * * root /root/bin/zfs-snapshot.sh tank hourly 25
</pre>
<p>And daily/weekly/monthly snapshots out of <code>/usr/local/etc/anacrontab</code> (from the sysutils/anacron port):</p>
<pre class="brush: bash; highlight: [9,11,13]; title: ; notranslate">
PATH=/bin:/sbin:/usr/bin:/usr/sbin

# days		make sure the command is executed at least every 'days' days
# delay		delay in minutes, before a command starts
# id		unique id of a command

# days	delay	id		command
1	5	daily		periodic daily
1	10	daily_snap	/root/bin/zfs-snapshot.sh tank daily 8
7	15	weekly		periodic weekly
7	30	weekly_snap	/root/bin/zfs-snapshot.sh tank weekly 5
30	60	monthly		periodic monthly
30	90	monthly_snap	/root/bin/zfs-snapshot.sh tank monthly 13
</pre>
<p>(Of course, this isn&#8217;t as cool as the Gnome-integrated <a href="http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs">Time</a> <a href="http://blogs.sun.com/erwann/entry/new_time_slider_features_in">Slider</a> in OpenSolaris, but it scratches my itch sufficiently.)</p>
]]></content:encoded>
			<wfw:commentRss>http://andyleonard.com/2010/04/07/automatic-zfs-snapshot-rotation-on-freebsd/feed/</wfw:commentRss>
		<slash:comments>11</slash:comments>
		</item>
		<item>
		<title>Duplicity to Amazon S3 on FreeBSD: Building on the work of others</title>
		<link>http://andyleonard.com/2009/03/02/duplicity-to-amazon-s3-on-freebsd-building-on-the-work-of-others/</link>
		<comments>http://andyleonard.com/2009/03/02/duplicity-to-amazon-s3-on-freebsd-building-on-the-work-of-others/#comments</comments>
		<pubDate>Mon, 02 Mar 2009 19:47:53 +0000</pubDate>
		<dc:creator>Andy</dc:creator>
				<category><![CDATA[freebsd]]></category>
		<category><![CDATA[storage]]></category>
		<category><![CDATA[aws]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[duplicity]]></category>
		<category><![CDATA[s3]]></category>

		<guid isPermaLink="false">http://andyleonard.com/?p=226</guid>
		<description><![CDATA[(This post adds only a couple small details to work described at randys.org and cenolan.com &#8211; go there for background on this post and useful scripts for automated Duplicity backup to S3.) First off, if you want to use Duplicity installed from FreeBSD Ports to backup to Amazon S3, be sure to also install the [...]]]></description>
			<content:encoded><![CDATA[<p>(This post adds only a couple small details to work described at <a href="http://www.randys.org/2007/11/16/how-to-automated-backups-to-amazon-s-s3-with-duplicity/">randys.org</a> and <a href="http://www.cenolan.com/2008/12/how-to-incremental-daily-backups-amazon-s3-duplicity/">cenolan.com</a> &#8211; go there for background on this post and useful scripts for automated Duplicity backup to S3.)</p>
<p>First off, if you want to use Duplicity installed from FreeBSD Ports to backup to Amazon S3, be sure to also install the <code>devel/py-boto</code> and <code>security/pinentry-curses</code> ports.</p>
<p>If you attempt to run the backup script described at randys.org or cenolan.com from cron, you may run into an error similar to the following:<br />
<span id="more-226"></span></p>
<pre>2009-03-01_01:05:05: ... backing up filesystem
Cleanup of temporary directory /tmp/duplicity-gM4CN9-tempdir failed - this
is probably a bug.
Cleanup of temporary directory /tmp/duplicity-gM4CN9-tempdir failed - this
is probably a bug.
Traceback (most recent call last):
File "/usr/local/bin/duplicity", line 583, in &lt;module&gt;
with_tempdir(main)
File "/usr/local/bin/duplicity", line 577, in with_tempdir
fn()
File "/usr/local/bin/duplicity", line 558, in main
full_backup(col_stats)
File "/usr/local/bin/duplicity", line 234, in full_backup
bytes_written = write_multivol("full", tarblock_iter, globals.backend)
File "/usr/local/bin/duplicity", line 148, in write_multivol
globals.gpg_profile, globals.volsize)
File "/usr/local/lib/python2.5/site-packages/duplicity/gpg.py", line 240,
in GPGWriteFile
bytes_to_go = data_size - get_current_size()
File "/usr/local/lib/python2.5/site-packages/duplicity/gpg.py", line 232,
in get_current_size
return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory:
'/tmp/duplicity-gM4CN9-tempdir/mktemp-iZknw0-2'

Traceback (most recent call last):
File "/usr/local/bin/duplicity", line 583, in &lt;module&gt;
with_tempdir(main)
File "/usr/local/bin/duplicity", line 577, in with_tempdir
fn()
File "/usr/local/bin/duplicity", line 558, in main
full_backup(col_stats)
File "/usr/local/bin/duplicity", line 232, in full_backup
sig_outfp = get_sig_fileobj("full-sig")
File "/usr/local/bin/duplicity", line 210, in get_sig_fileobj
fh = globals.backend.get_fileobj_write(sig_filename)
File "/usr/local/lib/python2.5/site-packages/duplicity/backend.py", line
354, in get_fileobj_write
fh = dup_temp.FileobjHooked(tdp.filtered_open("wb"))
File "/usr/local/lib/python2.5/site-packages/duplicity/path.py", line 716,
return gpg.GPGFile(1, self, gpg_profile)
File "/usr/local/lib/python2.5/site-packages/duplicity/gpg.py", line 112,
in __init__
'logger': self.logger_fp})
File "/usr/local/lib/python2.5/site-packages/GnuPGInterface.py", line 357,
in run
create_fhs, attach_fhs)
File "/usr/local/lib/python2.5/site-packages/GnuPGInterface.py", line 401,
in _attach_fork_exec
if process.pid == 0: self._as_child(process, gnupg_commands, args)
File "/usr/local/lib/python2.5/site-packages/GnuPGInterface.py", line 442,
in _as_child
os.execvp( command[0], command )
File "/usr/local/lib/python2.5/os.py", line 354, in execvp
_execvpe(file, args)
File "/usr/local/lib/python2.5/os.py", line 390, in _execvpe
func(fullname, *argrest)
OSError: [Errno 2] No such file or directory

Traceback (most recent call last):
File "/usr/local/bin/duplicity", line 583, in &lt;module&gt;
with_tempdir(main)
File "/usr/local/bin/duplicity", line 577, in with_tempdir
fn()
File "/usr/local/bin/duplicity", line 558, in main
full_backup(col_stats)
File "/usr/local/bin/duplicity", line 234, in full_backup
bytes_written = write_multivol("full", tarblock_iter, globals.backend)
File "/usr/local/bin/duplicity", line 148, in write_multivol
globals.gpg_profile, globals.volsize)
File "/usr/local/lib/python2.5/site-packages/duplicity/gpg.py", line 237,
in GPGWriteFile
file = GPGFile(True, path.Path(filename), profile)
File "/usr/local/lib/python2.5/site-packages/duplicity/gpg.py", line 112,
in __init__
'logger': self.logger_fp})
File "/usr/local/lib/python2.5/site-packages/GnuPGInterface.py", line 357,
in run
create_fhs, attach_fhs)
File "/usr/local/lib/python2.5/site-packages/GnuPGInterface.py", line 401,
in _attach_fork_exec
if process.pid == 0: self._as_child(process, gnupg_commands, args)
File "/usr/local/lib/python2.5/site-packages/GnuPGInterface.py", line 442,
in _as_child
os.execvp( command[0], command )
File "/usr/local/lib/python2.5/os.py", line 354, in execvp
_execvpe(file, args)
File "/usr/local/lib/python2.5/os.py", line 390, in _execvpe
func(fullname, *argrest)
OSError: [Errno 2] No such file or directory</pre>
<p>The solution to the above is simple &#8211; make sure the path includes <code>/usr/local/bin</code>, perhaps by including this at the start of the backup script:</p>
<pre>export PATH=${PATH}:/usr/local/bin</pre>
<p>Finally, when running an incremental backup, you may get this error:</p>
<pre>Fatal Error: Neither remote nor local manifest is readable.</pre>
<p>This can be solved by setting the <code>HOME</code> environment variable to <code>/root</code> assuming you&#8217;re running the backup as root (instead of the default <code>/var/log</code> for cron jobs):</p>
<pre>export HOME=/root</pre>
]]></content:encoded>
			<wfw:commentRss>http://andyleonard.com/2009/03/02/duplicity-to-amazon-s3-on-freebsd-building-on-the-work-of-others/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
	</channel>
</rss>

