This is a picture of me, Andrew Forward Technical Tidbits

A collection of code snippets of sometimes rarely used, or perhaps difficult to remember tasks, commands and code snippets.

Changing a YAML file in Ruby

require "yaml"
File.open("./example.yml", 'w') {|f| f.write("
workbook: 5
sheet: 10
header_index: 5
headers:
  - hello
  - hi
  - bye
") }
data = YAML.load_file("./example.yml")
puts "OLD DATA: #{data}"

data["headers"] = [ "Hello", "Hi", "Bye" ]
File.open("./example.yml", 'w') {|f| f.write(data.to_yaml) }

data = YAML.load_file("./example.yml")
puts "NEW DATA: #{data}"

Clean bundler within a Ruby Script

oenv = ENV.to_hash
begin
  %w{BUNDLE_GEMFILE RUBYOPT GEM_HOME GIT_DIR GIT_WORK_TREE BUNDLE_BIN_PATH}.each { |key| ENV.delete(key) }
  output = IO.popen("xlshell "#{filename}" "#{worksheet}"")
  return output.readlines.collect { |f| f.strip }
ensure
  ENV.replace(oenv)
end

Clear Your Local DNS Cache

dscacheutil -flushcache

On newer Macs, somethings this works too.

sudo killall -HUP mDNSResponder

Create a public / private key

ssh-keygen -t rsa

Dealing with CSV files and MS Excel (.xls / .xlsx)

When dealing with CSV, you need to get your encodings correct.Excel will typically export to ISO-8859-1. This is not good, and should be converted to UTF-8 to support non english files.

To check the encoding of a file

file -I original_file.csv

To convert between encodings (a few common examples)

iconv --from-code=ISO-8859-1 --to-code=UTF-8 original_file.csv > new_file.utf8.csv
iconv --from-code=MacRoman --to-code=UTF-8 original_file.csv > new_file.utf8.csv

When exporting files to be read within Excel, you will need to convert the CSV based on the target platform.

For MAC

iconv --from-code=UTF-8 --to-code=MacRoman downloaded_file.csv > downloaded_file_mac.excel.csv

For PC (untested)

iconv --from-code=UTF-8 --to-code=Windows-1252 downloaded_file.csv > downloaded_file_windows.excel.csv

Deleting Git Branches

Delete the "demo_maintenance" branch on the "origin" remote server.

git push origin :demo_maintenance # deletes the remote branch
git branch -d demo_maintenance # deletes the local branch

If you need to 'force' the delete on your local machine, use -D

git branch -D demo_maintenance # deletes the local branch

Delete Rails sessions

Remove all old sessions, and then optimize the table.</p>

DELETE
FROM sessions
WHERE updated_at < DATE_SUB(NOW(), INTERVAL 1 DAY);

optimize TABLE sessions;

Destroy All RabbitMQ Queues

Should only be done on a dev machine:

rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl start_app

Faster Macbook Pro Wakeups

A longer discussion at ewal.net

sudo pmset -a standbydelay 86400 # only standby after 24 hours

Find text in files within a directory

grep -r "some words reward" /path/tp/look/*

Find large files on a Linux / Mac OS X machine

The following should owrk on your Mac OS X

sudo find / -type f -size +20000k -exec ls -lh {} ; | awk '{ print $9 ": " $5 }'

For a linux (tested on Ubuntu), the $9 most liley needs to be an $8

sudo find / -type f -size +20000k -exec ls -lh {} ; | awk '{ print $8 ": " $5 }'

Enable FTP on Mac OS X Lion

# enable the ftp server
sudo -s launchctl load -w /System/Library/LaunchDaemons/ftp.plist

#disable the server when you are done
sudo -s launchctl unload -w /System/Library/LaunchDaemons/ftp.plist

Checking Out New Git Branches

First locat the branches, then checkout the one you want (e.g. origin/demo1)

git branch -r
git checkout --track -b demo1 origin/demo1

Undo a local commit

If you committing a git change locally, you can undo it using the following command:

git reset --soft HEAD^

Undo the last pushed commited

If you already committed the change, then you need to do hard resets:

git reset --hard HEAD~1
git push -f

Granting passwordless access to root on RHEL

When running CHEF, it needs root access, so if you are using something like chef-solo, and you like automated scripts, then here is how grant that access. PLEASE NOTE, THIS IS VERY UNSAFE, to avoid try, I created a (now obsolete) CHEF BOOTSTRAP.

$ mkdir -p /root/.ssh
$ chmod 700 /root/.ssh
$ chmod 600 /roiot/.ssh/authorized_keys
$ restorecon -R -v /root/.ssh

Install older version of HTML to PDF (wkhtmltopdf) using Homebrew on a Mac

Bug with current version that will cause the process to not exit properly:</p>

brew uninstall wkhtmltopdf
cd /usr/local/Library/Formula/
git checkout 6e2d550 /usr/local/Library/Formula/wkhtmltopdf.rb
brew install wkhtmltopdf
wkhtmltopdf --version | grep 0.9.9

Ticket was described here, but now the site (and history) have for wkhtmltopdf.org have moved. Thank you to wearepandr

Kill a Defunct Process

# Locate defunct processes
ps -A | grep defunct

# Get the parent ID (UID PID PPID ...)
ps -ef | grep defunct | more

# for example...
deployer  2707  2698  0  2012 ?        00:00:03 [unicorn_rails] <defunct>
deployer  2710  2698  0  2012 ?        00:00:04 [unicorn_rails] <defunct>

# Kill the parent PID (PPID)
kill -9 2698

See how much memory something consumes in PHP

$start_time = microtime(true);
$start_memory = memory_get_usage();

// INSERT CODE HERE

$end_memory = memory_get_usage();
$end_time = microtime(true);

print_r(array(
  'memory (Mb)' => ($end_memory - $start_memory) / (1024 * 1024),
  'time (s)' => $end_time - $start_time
));

Memcache in PHP

$memcache = new Memcache();
$memcache->connect('localhost', 11211);

function doSomethingSlow() {
  sleep(5);
  return "foobar";
}

$name = $memcache->get('name');
if (!$name)
{
  $name = doSomethingSlow();
  $memcache->set('name',$name);
}

echo $name;

Execute a single MySQL statement from command line

mysql -u myname -pmypassword -D mydb -e "show tables"

Granting access to mysql users

SHOW GRANTS FOR deployer
# ALL ACCESS
GRANT ALL ON myshop.* TO 'deployer'@'%' IDENTIFIED BY 'pass' ;
# SELECT ONLY, and filtered by IP
GRANT SELECT ON myshop.* TO deployer@'192.168.1.%' IDENTIFIED BY 'pass';
# Or by domain name
GRANT SELECT ON myshop.* TO deployer@'webapp.myserver.com' IDENTIFIED BY 'pass';
FLUSH PRIVILEGES;

Removing access to mysql users

REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'deployer'@'%';

How to change owners of a postgres database

GRANT ALL PRIVILEGES ON DATABASE @DATABASE_NAME@ TO @NEW_USERNAME@;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO @NEW_USERNAME@;

For example, changing from the 'postgres' user to the 'deployer' user.

GRANT ALL PRIVILEGES ON DATABASE my_app TO deployer;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO deployer;

Granting access to postgres users

CREATE USER myuser  WITH ENCRYPTED PASSWORD 'ppp';
GRANT USAGE ON SCHEMA public to myuser;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO myuser;

GRANT CONNECT ON DATABASE mydb to myuser;
c mydb
GRANT USAGE ON SCHEMA public to myuser;
GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO myuser;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO myuser;

Clean / Clear / Delete Rails Databse Sessions

Don't forget to clear your sessions every once in a while

rake db:sessions:clear

Run A Script Remotely Using SSH

For simple scripts, just pass them via a quoted string.

ssh name@server.com 'ls -la'

For more complicated scripts, put them in a local file and stream them to the server.

ssh name@server.com 'bash -s' < local_script.sh

Rename exentions on a file

find . -name '*.php.eex' -exec sh -c 'mv "$0" "${0%.php.eex}.eex"' {} ;
find . -name '*.php' -exec sh -c 'mv "$0" "${0%.php}.eex"' {} ;

Maintain should support in latest rspec gem

If you receive warnings about should and either are not ready to change, or don't want to, here is how you can silence the warnings.

Using should from rspec-expectations' old :should syntax without explicitly enabling the syntax is deprecated. Use the new :expect syntax or explicitly enable :should instead.

Using stub from rspec-mocks' old :should syntax without explicitly enabling the syntax is deprecated. Use the new :expect syntax or explicitly enable :should instead.

RSpec.configure do |config|
  config.mock_with :rspec do |c|
    c.syntax = [:should, :expect]
  end
  config.expect_with :rspec do |c|
    c.syntax = [:should, :expect]
  end
end

Tar / Untar a file

Compress a directly into a tar.gz file:

tar cvzf myapp.tar.gz myapp/

Uncompress that file back into a directory:

tar zxfv myapp.tar.gz

Uncompress a gz zip (no tar):

gzcat x.txt.gz >x.txt
gunzip -c x.txt.gz >x.txt

Which Port is a Process Running On

Find which process is listening on a certain port

lsof -i :12345 # for port 12345