Setting up secure backups

I’ve decided to move my online storage off Dropbox and as part of this plan, my backups will also now be real backups. I’m moving to SpiderOak for numerous reasons, one of which is the zero knowledge encryption. The other is they like and support Linux, which is a huge plus and really a necessity. They’re also a backup company first, instead of a storage company. Those who know understand what I mean.

https://spideroak.com/download/referral/caced444a30fc855b5da8c1b683bff19

Like always, if you use my referral link it helps me and you. So please do! :)

Scrape site for inventory

While looking for a new carry gun I decided to just scrape the site every morning and email myself the inventory level.

 

root@mail:~# cat scraper.sh 
#!/bin/bash
 
inventory=`curl -s http://www.shootersconnectionstore.com/STI-40-Duty-One-9x19-Lite-Night-Sights-P3225.aspx | grep inventoryDetails`
 
SUBJECT="STI Duty One Inventory"
EMAIL="myemail@gmail.com"
 
echo -e $inventory | mail -a "From: info@mydomain.com" -a "Content-Type: text/html" -s "$SUBJECT" "$EMAIL"

New Peruta Order

In an unexpected order given by the court the State of California and the Brady Campaign have been given,

Each party is each directed to file a response of no more than 6,000 words

addressing the pending motions to intervene filed with this Court on February 27,

Download30 downloads

PHP rDNS CSV Generator

Quick script to generate rDNS for the Bluecat Proteus IPAM import.

<?php
 
$minhost=4;
$maxhost=126;
$recordtype = "PTR";
$action = "add";
$arpa = ".10.168.192.in-addr.arpa.";
$ttl = "";
 
if (file_exists("ptr_records.csv")) die ("File already exists\n");
 
$file = fopen("ptr_records.csv", 'w') or die ("Failed to create file\n");
 
for($count = $minhost; $count <= $maxhost; $count++) {
        $text = $action . "," . $count . $arpa . "," . $ttl . "," . $recordtype  . ",192.168.10." . $count . ".domain.com\n";
        file_put_contents("ptr_records.csv",$text,FILE_APPEND);
}
 
fclose($file);
echo "File created\n";
 
?>

AUP Script

This script looks for a string over HTTP in a subnet supplied in CIDR notation

#!/bin/bash
 
function addzeros {
   NUM=$1
   ZS=$2
   NBITS=$(echo "$NUM" | grep -o "[0-9]" | wc -l)
   if (( $NBITS < $ZS )); then
      ZEROS2ADD=$(expr $ZS - $NBITS)
      while (( $ZEROS2ADD << 0 )); do
         NUM="0$NUM"
         ZEROS2ADD=$(expr $ZEROS2ADD - 1)
      done
      echo "$NUM"
   else
      echo $NUM
   fi
}
 
function bin2dec {
   echo "obase=10; ibase=2; $1" | bc
}
function dec2bin {
   echo "obase=2; $1" | bc
}
 
function ipbin2ip {
   cont=1
   for l in $(echo $1 | grep -o [0,1]); do
      VIPTMP[$cont]=$l
      cont=$(expr $cont + 1)
   done
   ONE=`bin2dec $(echo ${VIPTMP[@]:1:8} | tr -d [:space:])`
   TWO=`bin2dec $(echo ${VIPTMP[@]:9:8} | tr -d [:space:])`
   THREE=`bin2dec $(echo ${VIPTMP[@]:17:8} | tr -d [:space:])`
   FOUR=`bin2dec $(echo ${VIPTMP[@]:25:8} | tr -d [:space:])`
   FINALIP=$(echo ${ONE}.${TWO}.${THREE}.${FOUR})
   CONTENT=$(curl -L -m 2 -s $FINALIP| grep -i $SEARCH)
   if [ -z "$CONTENT" ]; then
	echo -n .
   else
	echo $FINALIP" -> "$CONTENT
   fi
}
 
if [ -z $1 ]; then
	echo "Usage: cidrSearch <CIDR> <Search String>"
	echo "Example: cidrSearch 74.125.224.195/30 title"
	exit 0
fi
 
SEARCH=$2
IPCIDR=$1
IP=$(echo $IPCIDR | cut -d"/" -f1)
BITS=$(echo $IPCIDR | cut -d"/" -f2)
 
FST=$(echo $IP | cut -d"." -f1)
FSTBIN=`addzeros $(echo "obase=2; $FST" | bc) 8`
SND=$(echo $IP | cut -d"." -f2)
SNDBIN=`addzeros $(echo "obase=2; $SND" | bc) 8`
TRD=$(echo $IP | cut -d"." -f3)
TRDBIN=`addzeros $(echo "obase=2; $TRD" | bc) 8`
FOH=$(echo $IP | cut -d"." -f4)
FOHBIN=`addzeros $(echo "obase=2; $FOH" | bc) 8`
 
VIP=0.0.0.0
IPBIN="$FSTBIN$SNDBIN$TRDBIN$FOHBIN"
 
i=1
for l in $(echo $IPBIN | grep -o [0,1]); do
   VIPBIN[$i]="$l"
   i=$(expr $i + 1)
done
 
BITSHOST=$(expr 32 - $BITS)
i=$BITSHOST
while (( $i > 0 )); do
   MAXHOSTBIN=1$MAXHOSTBIN
   i=$(expr $i - 1)
done
MAXHOST=$(bin2dec $MAXHOSTBIN)
 
n=$(echo ${VIPBIN[@]:1:$BITS} | tr -d [:space:])
c=$MAXHOST
while (( $c > 0 )); do
   h=$c
   EUREKA=${n}$(addzeros `dec2bin $h` $BITSHOST)
   ipbin2ip "$EUREKA"
   c=$(expr $c - 1)
done

DNS Query Script

We had a customer complaining about their IP address not resolving and slow lookup times. I ended up scripting the lookup which does a dig on the domain and retains the first A record IP and then on subsequent lookups compares the original IP with the one just looked up. Each time the script does a lookup it outputs the query time (similar to ping). When the script detects the IP as having changed, it will exit and print a summary.

Disclaimer:
This is not useful for people doing round-robin DNS balancing.

#!/bin/bash
 
trap ctrl_c INT
 
function ctrl_c() {
 
	find_mean
	printf "\n"
	echo Checked $counter times
	printf "\n"
	echo High: $mshigh ms  Low: $mslow ms  Avg: $msmean ms
	exit
}
 
function find_mean() {
	let "msmean = ($mstotal / $counter)"
}
 
full=`dig $1`
initialIp=`echo "$full" | awk '/ANSWER SECTION/ { getline; print $0 }' | awk '{print $5}'`
ip=$initialIp
 
counter=0
 
while [ "$initialIp" = "$ip" ]; do
	full=`dig $1`
	ip=`echo "$full" | awk '/ANSWER SECTION/ { getline; print $0 }' | awk '{print $5}'`
	msprint=`echo "$full" | awk ' /Query/ {print $4,$5}'`
	mskeep=`echo "$full" | awk ' /Query/ {print $4}'`
 
	if [ -z $mslow ]
		then
			mslow=$mskeep
	fi
	if [ -z $mshigh ]
		then
			mshigh=$mskeep
	fi
	if [ $mskeep -lt  $mslow ]
		then
			mslow=$mskeep
	fi
	if  [ $mskeep -gt $mshigh ]
		then
			mshigh=$mskeep
	fi
 
	let mstotal+=$mskeep
	echo $msprint
	let counter+=1
	sleep 15
done
 
find_mean
echo Checked $counter times
printf "\n"
echo Found discrepancy: Initial IP was $initialIp and the IP is now $ip
printf "\n"
echo High: $mshigh ms  Low: $mslow ms  Avg: $msmean ms

Lucene Search Init Script – CentOS6

This is the init script that used for CentOS 6 and Apache’s Lucene search for our Mediawiki implementation.

 

#!/bin/bash
# chkconfig: 2345 90 90
# description: program_name
### BEGIN INIT INFO
# Provides: program_name
# Required-Start: network
# Required-Stop: network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: Start the program
### END INIT INFO
 
### Fill in these bits:
START_CMD="/usr/local/search/ls2/lsearchd"
NAME="search"
PGREP_STRING="/usr/local/search/ls2/lsearchd"
PID_FILE="/var/run/search/search.pid"
USER="root"
 
### No further muckin' about needed!
 
CUR_USER=`whoami`
 
killproc() {
  pkill -u $USER -f $PGREP_STRING
}
 
start_daemon() {
  eval "$*"
}
 
log_success_msg() {
  echo "$*"
  logger "$_"
}
 
log_failure_msg() {
  echo "$*"
  logger "$_"
}
 
check_proc() {
  pgrep -u $USER -f $PGREP_STRING >/dev/null
}
 
start_script() {
  if [ "${CUR_USER}" != "root" ] ; then
    log_failure_msg "$NAME can only be started as 'root'."
    exit -1
  fi
 
  check_proc
  if [ $? -eq 0 ]; then
    log_success_msg "$NAME is already running."
    exit 0
  fi
 
  [ -d /var/run/$NAME ] || (mkdir /var/run/$NAME )
 
   # make go now 
    start_daemon /bin/su $USER -c $START_CMD >> /dev/null &
 
  # Sleep for a while to see if anything cries
  sleep 5
  check_proc
 
  if [ $? -eq 0 ]; then
    log_success_msg "Started $NAME."
  else
    log_failure_msg "Error starting $NAME."
    exit -1
  fi
}
 
stop_script() {
  if [ "${CUR_USER}" != "root" ] ; then
    log_failure_msg "You do not have permission to stop $NAME."
    exit -1
  fi
 
  check_proc
  if [ $? -eq 0 ]; then
    killproc -p $PID_FILE >/dev/null
 
    # Make sure it's dead before we return
    until [ $? -ne 0 ]; do
      sleep 1
      check_proc
    done
 
    check_proc
    if [ $? -eq 0 ]; then
      log_failure_msg "Error stopping $NAME."
      exit -1
    else
      log_success_msg "Stopped $NAME."
    fi
  else
    log_failure_msg "$NAME is not running or you don't have permission to stop it"
  fi
}
 
check_status() {
  check_proc
  if [ $? -eq 0 ]; then
    log_success_msg "$NAME is running."
  else
    log_failure_msg "$NAME is stopped."
    exit -1
  fi
}
 
case "$1" in
  start)
    start_script
    ;;
  stop)
    stop_script
    ;;
  restart)
    stop_script
    start_script
    ;;
  status)
    check_status
    ;;
  *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
esac
 
exit 0

Lsyncd Configuration

This is the configuration I used for keeping the images synced across two Mediawiki servers. I retained the default 5 second delay after the WRITE_CLOSE is called.

settings {
        logfile    = "/var/log/lsyncd/lsyncd.log",
        statusFile = "/var/log/lsyncd/lsyncd.status",
}
 
sync {
        default.rsyncssh,
        source = "/var/www/html/images/",
        host = "10.101.2.99",
        targetdir = "/var/www/html/images/",
        rsync = {
                compress = false,
                links = true,
                verbose = true,
        }
}

Lsyncd Init Script – CentOS6

The RPM on CentOS 6 for Lsyncd has a bad init script. I was unable to get the daemon to run properly for what seemed like unknown reasons. Instead of debugging the script I just used this generic one which works. Once I get some time I’ll go through the one supplied in the RPM and file a bug report.

#!/bin/bash
#
# lsyncd: Starts the lsync Daemon
#
# chkconfig: 345 80 30
# description: Lsyncd uses rsync to synchronize local directories with a remote
# machine running rsyncd. Lsyncd watches multiple directories
# trees through inotify. The first step after adding the watches
# is to, rsync all directories with the remote host, and then sync
# single file buy collecting the inotify events.
# processname: lsyncd
# config: /etc/lsyncd.conf
# pidfile: /var/run/lsyncd.pid
 
# Source function library.
. /etc/init.d/functions
 
RETVAL=0
PIDFILE="/var/run/lsyncd.pid"
LOCKFILE="/var/lock/subsys/lsyncd"
LSYNCD="/usr/bin/lsyncd"
CONFIG="/etc/lsyncd.conf"
PROG="lsyncd"
 
start() {
        echo -n "Starting $PROG: "
 
        if [ -f $PIDFILE ]; then
                PID=`cat $PIDFILE`
                echo $PROG already running: $PID
                exit 1;
        else
                daemon --pidfile=$PIDFILE $LSYNCD -pidfile $PIDFILE $CONFIG
                RETVAL=$?
                echo
                [ $RETVAL -eq 0 ] &amp;&amp; touch $LOCKFILE
                return $RETVAL
        fi
 
}
 
stop() {
        echo -n "Stopping $PROG: "
 
        killproc lsyncd
        echo
        rm -f $LOCKFILE
        return 0
 
}
 
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status lsyncd
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "Usage:  {start|stop|status|restart}"
        exit 1
        ;;
esac
exit $?