We have extracted ActivityStreams from Legaltorrents.com and released it under the BSD license.

This is a new plugin for the Rails Community that provides a customizable framework for cataloging and publishing user activity and social objects. We currently aim to provide support for microformats in HTML, Atom feeds, and compatibility with the open source DISO social networking implementation for activity discovery and consumption.

Here is a quickie for Rails 2.1:

./script/plugin install git://github.com/face/activity_streams.git

It may work on earlier versions of Rails as well. Don’t have git:

./script/plugin install svn://rubyforge.org/var/svn/activitystreams/trunk

For further documentation here is the rdoc and the AcitivityStreams Home Page, or just show me the code.

development:
  adapter: mysql
  database: legaltorrents_development
  username: fred
  password: password
  socket: /tmp/mysql.sock

production:
  adapter: postgresql
  database: legaltorrents_production
  username: fred
  password: password
  host: localhost

Thanks to Ruby on Rails, transferring and converting database from one database platform to another only takes a few lines of code. There are rake tasks for dumping to YAML and back. However the existing YAML scripts I found had issues with some of our data and then failed for blobs. This script will only work with a “rails style” database. By “rails style” I mean any database where every table has a unique key named “id”.

A special thanks to Matson Systems, Inc. for having me write this script for LegalTorrents and then contribute it to all under a BSD license.

1
2
3

rake db:migrate
env RAILS_ENV=production rake db:migrate

1
2
3

cd lib/tasks
wget 'http://github.com/face/rails_db_convert_using_adapters/tree/master%2Fconvert.rake?raw=true' -O convert.rake

Update Oct 1, 2008:Fixed a bug today for Rails 2.1.1. Also got rid of the hash of data that was a relic from an early version of the script that used a single model object.

Here is the full code to convert.rake:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115

#
# Convert/transfer data from production => development.    This facilitates
# a conversion one database adapter type to another (say postgres -> mysql )
#
# WARNING 1: this script deletes all development data and replaces it with
#                     production data
#
# WARNING 2: This script assumes it is the only user updating either database.
#                     Database integrity could be corrupted if other users where 
#                     writing to the databases.
#
# Usage:  rake db:convert:prod2dev
#
# It assumes the development database has a schema identical to the production 
# database, but will delete any data before importing the production data
#
# A couple of the outer loops evolved from 
#    http://snippets.dzone.com/posts/show/3393
#
# For further instructions see 
#    http://myutil.com/2008/8/31/rake-task-transfer-rails-database-mysql-to-postgres
#
# The master repository for this script is at github:
#    http://github.com/face/rails_db_convert_using_adapters/tree/master
#
#
# Author: Rama McIntosh
#         Matson Systems, Inc.
#         http://www.matsonsystems.com
#
# This rake task is released under this BSD license:
#
# Copyright (c) 2008, Matson Systems, Inc. All rights reserved.
# 
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 
# * Redistributions of source code must retain the above copyright
#   notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
#   notice, this list of conditions and the following disclaimer in the
#   documentation and/or other materials provided with the distribution.
# * Neither the name of Matson Systems, Inc. nor the names of its
#   contributors may be used to endorse or promote products derived
#   from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

# PAGE_SIZE is the number of rows updated in a single transaction.
# This facilitates tables where the number of rows exceeds the systems
# memory
PAGE_SIZE=10000

namespace :db do
  namespace :convert do    
    desc 'Convert/import production data to development.   DANGER Deletes all data in the development database.   Assumes both schemas are already migrated.'
    task :prod2dev => :environment do

      # We need unique classes so ActiveRecord can hash different connections
      # We do not want to use the real Model classes because any business
      # rules will likely get in the way of a database transfer
      class ProductionModelClass < ActiveRecord::Base
      end
      class DevelopmentModelClass < ActiveRecord::Base
      end

      skip_tables = ["schema_info", "schema_migrations"]
      ActiveRecord::Base.establish_connection(:production)
      (ActiveRecord::Base.connection.tables - skip_tables).each do |table_name|

        ProductionModelClass.set_table_name(table_name)
        DevelopmentModelClass.set_table_name(table_name)
        DevelopmentModelClass.establish_connection(:development)
        DevelopmentModelClass.reset_column_information
        ProductionModelClass.reset_column_information
        DevelopmentModelClass.record_timestamps = false

        # Page through the data in case the table is too large to fit in RAM
        offset = count = 0;
        print "Converting #{table_name}..."; STDOUT.flush
        # First, delete any old dev data
        DevelopmentModelClass.delete_all
        while ((models = ProductionModelClass.find(:all, 
            :offset=>offset, :limit=>PAGE_SIZE)).size > 0)

          count += models.size
          offset += PAGE_SIZE

          # Now, write out the prod data to the dev db
          DevelopmentModelClass.transaction do
            models.each do |model|
              new_model = DevelopmentModelClass.new(model.attributes)
              new_model.id = model.id
              new_model.save(false)
            end
          end
        end
        print "#{count} records converted\n"
      end
    end
  end
end

As some of you know I’ve been working full time for Matson Systems, Inc. building out LegalTorrents. I must apologize I have been neglecting parts of my blog. Fortunately, I’ve been swamped building out cool features for LegalTorrents and Matson wants to contribute back. After this caching article look for a rake task to convert a Rails app from one database platform to another, then a plugin for generating Activity Streams.

As a new community, we have to support tens of thousands of registered users. Yet on any given news day we need to support hundreds of thousands of non registered users.

Let’s begin with conditional caching in Rails 2.1 (if 2.0, see below). Conditional action caching is a new feature of the Rails 2.1 API. First off, pre Rails 2.1 the default was disk. In rails 2.1, the default is RAM. Not going to work on limited resources:

1
2
3

# Put this in RAILS_ROOT/config/initializers/something.rb
ActionController::Base.cache_store = :file_store, "#{RAILS_ROOT}/tmp/cache"

1
2
3
4
5
6
7
8

class ApplicationController < ActionController::Base
  # ...
  protected
  # of course logged_in? needs to be defined...restful_authentication is what I recommend.
  def do_caching?
    !logged_in? && flash.empty?
  end

1
2
3
4
5
6

class TorrentsController < ApplicationController
  # ...
  caches_action :show , :if => Proc.new { |controller|
                          controller.send(:do_caching?) }

1
2
3

# This cron entry that runs every 10 minutes and removes any files older than 10 minutes named '*.cache'
3,13,23,33,43,53 * * * * find /home/ltdeploy/legaltorrents/tmp/cache -mmin +10 -name '*.cache' -exec rm -f {} \;

1
2
3
4
5
6
7
8
9
10

class CategoriesController < ApplicationController

  caches_action :show, 
    :if => Proc.new { |controller| controller.send(:do_caching?) }, 
    :cache_path => Proc.new { |c|
      c.params[:page] ?
      "#{c.request.host}.#{c.request.port}/#{c.send(:category_path,c.params[:id])}/page/#{c.params[:page]}" :
      "#{c.request.host}.#{c.request.port}/#{c.send(:category_path,c.params[:id])}/page/1"
    }

End of Story for Rails 2.1

Now, Rails 2.0 doesn’t have :if in caches_action. To work around this we used a simple monkey patch:

1
2
3
4
5
6
7
8

class ApplicationController < ActionController::Base
  # ...
  protected
  # Overrides Rails core to do action_cache when not logged in...Only works in Rails 2.0 and maybe earlier
  def perform_caching
    @@perform_caching && !logged_in? && flash.empty?
   end

1
2
3
4

class TorrentsController < ApplicationController
  # ...
  caches_action :show

I was so excited, Apple and AT&T approved tethering for my iPhone…but it was gone, no wait, gizmodo say’s it’s back! (click on picture to the left which is the “App Store” link in the gizmodo article to install NetShare right now via iTunes). Update:The link is broken now, NetShare is not available in the US app store. For us lucky few who purchased it….it still works…

However, after installing NetShare via iTunes onto my iphone I could not get it to work. I followed the instructions only to get:

Safari can’t open the page. Safari can’t open the page “http://www.apple.com/”. The error was: “unknown error” (NSURLErrorDomain:-1) Please choose Report Bugs to Apple from the Safari menu, note the error number, and describe what you did before you saw this message.

After fiddling with it for a few minutes I found a solution. The iphone appeard to be still trying to using the WiFi connection and not 3G.

So, right before the final step of launching NetShare on the iphone (i.e. after setting up the adHoc network on the iphone to your PC), launch Safari on the iPhone and surf to a page. This will cause the little WiFi icon to change to 3G at the top of your phone. Now lanuch NetShare that you have the 3G icon back and everything will work like a dream.

I can even use the ssh client on my OSX laptop to connect to remote servers using connect.c to proxy ssh through the iphone. To get this to work was something like:

1
2
3
4
5
6

wget http://www.taiyo.co.jp/~gotoh/ssh/connect.c
gcc connect.c -o connect -lresolv
sudo cp connect /usr/local/bin/.
ssh -o 'ProxyCommand /usr/local/bin/connect -5 -S 10.10.10.1:1080 %h %p' 206.71.190.75
# Of course the ProxyCommand can go in your ~/.ssh/config

Have iPhone, Have Laptop, will travel!

OT – I’ve been doing most development on Ubuntu and OpenBSD. I recently made the mistake of ordering a fully loaded Dell Inspiron 1720, only to learn that Dell dumbed it down by replacing the DVI port with VGA and disabling the RAID support on the mother board. Instead of following Dell’s marketing plan and getting their XPS, I promptly went to Apple.com and bought a Macbook pro. I recommend development on the Macbook pro and Leopard is a great Unix development environment.

Okay, back on topic….My favorite solution to adding back the bookmark icons was a comment by klugerama in the comments of Lifehacker’s article Mac Tip: Add favicons to the Firefox bookmark toolbar

1
2
3
4
5

/* Kill(display: none) or show (display: inline) bookmark icons in the Personal Toolbar */ 
  toolbarbutton.bookmark-item > .toolbarbutton-icon {
  display: inline !important;
}

cd ~/Library/Application\ Support/Firefox/Profiles/c25zk8xx.default/chrome/
cp userChrome-example.css userChrome.css

1
2
3
4
5
6

/* change space around bookmark toolbar icons */
  #personal-bookmarks toolbarbutton {
    margin-left: -3px !important;
    margin-right: -3px !important;
}

1
2
3
4
5

/* Hide the Folders in the Personal Toolbar */
  toolbarbutton.bookmark-item[container] > .toolbarbutton-icon {
  display: none !important;
} 

I have a feeling this will help some of y’all if you are getting the following error:

Sorry! Something is not quite right with the request we received from the website you are trying to use. Please try again in a few minutes. If this error persists, please contact the site administrator for the website you are trying to use. If you are the site administrator, click here to contact us.

I get this error if I try to login on my development environment because localhost:3000 just won’t cut it for Yahoo’s OpenID security policy. If I run from a production URL on port 80, say http://myutil.com/ then signin works (though I haven’t gotten Simple Registration Attribute Exchange working with Yahoo).

Yahoo! Security Policies Yahoo! will only support Relying Parties running on webservers with real hostnames (IP addresses are not supported) running on standard ports (Port 80 for HTTP and Port 443 for HTTPS).

Hope this saves ya some time!

Update July 16th, 2008: Sphinx sphinx-0.9.8 has been released and this port has been updated.

Here is a OpenBSD port of Sphinx, Free open-source SQL full-text search engine. Sphinx is a wonderfully fast and memory efficient deep text search engine. I have found integrates nicely with Ruby and Ruby-On-Rails.

I hope at some point to get this port committed to the OpenBSD CVS repository. In the meantime this will be the ports home. Till it hits the OpenBSDs repository, it will live in my SVN repository here.

You can download a snapshot of the source code for the port here: sphinx.tgz. So far this port has been tested with MySQL and Postgresql on OpenBSD 4.2 and OpenBSD 4.3-betaOpenBSD-4.4-beta on the i386 platform with the Ultrasphinx Ruby-On-Rails plugin.

Instructions to build the port

If your ports tree is not already prepped, please begin by prepping your ports tree.

1
2
3
4
5
6

sudo su
export PKG_PATH="ftp://ftp2.usa.openbsd.org/pub/OpenBSD/4.3/packages/i386"
# or, setenv PKG_PATH "ftp://ftp2.usa.openbsd.org/pub/OpenBSD/4.3/packages/i386"
pkg_add libiconv mysql-server mysql-client
exit

Now we are ready to build the sphinx port:

1
2
3
4
5
6

cd /usr/ports/textproc
lynx --source "http://myutil.com/ports/4.4/sphinx.tgz" > sphinx.tgz
tar xvfz sphinx.tgz 
cd sphinx
make install  # or make package

1
2

env FLAVOR=pgsql make install

Thanks, and feedback is always welcome!

1
2
3
4
5
6
7
8
9
10
11

--- extconf.rb.orig     Tue Feb 26 12:03:53 2008
+++ extconf.rb  Tue Feb 26 11:52:16 2008
@@ -1,5 +1,7 @@
 require "mkmf"
 
+$LIBS += " -lstdc++ -laspell"
+
 have_header("ruby.h")
 have_header("aspell.h")
 have_library("aspell")

I recently installed the raspell gem on my development box as part of an evaluation of ultrasphinx.

Unfortunately, the native extension to raspell doesn’t link in the dynamic libraries it uses. A fix is provided below.

Note, using the raspell with ultrasphinx also caused a ruby core dump, this time in an assertion from the configuration for aspell. OpenBSD uses aspell-0.50.5 and after upgrading aspell to 0.60.5, raspell started working fine.

** Starting Rails with development environment...
ruby:/usr/local/lib/ruby/gems/1.8/gems/raspell-1.1/lib/raspell.so: undefined symbol 'new_aspell_config'
lazy binding failed!
/home/face/urevz/vendor/plugins/ultrasphinx/lib/ultrasphinx/spell.rb:33: [BUG] Segmentation fault
ruby 1.8.6 (2007-09-24) [i386-openbsd4.2]

A little poking around and I noticed the native library, raspell.so, doesn’t link in libaspell!

A quick fix is to add the libs to extconf.rb and rebuild the library:

1. cd /usr/local/lib/ruby/gems/1.8/gems/raspell-1.1/ext
2. edit extconf.rb and add the line $LIBS = ” -lstdc+ -laspell”
3. sudo make clean
4. sudo make
5. sudo cp raspell.so ../lib/.

Here is how to install i386 binaries (please choose an OpenBSD mirror near you…and please buy a t-shirt):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

sudo su

#for ksh/bash
export PKG_PATH="ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.2/packages/i386"

#or, for tcsh: setenv PKG_PATH=ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.2/packages/i386

# Now for Ruby
pkg_add "http://rubyforge.org/frs/download.php/32289/ruby-1.8.6.111.tgz"

# Optional, but Recommend for Rails
pkg_add "http://rubyforge.org/frs/download.php/32290/ruby-gems-1.0.1.tgz"
pkg_add "http://rubyforge.org/frs/download.php/32291/ruby-iconv-1.8.6.111.tgz"

# Now you could install rails if you wanted:
gem install rails

Rubyforge doesn’t allow you to have the same filename, even if it is in a different sub-package and release. Therefore, amd64 and sparc64 binaries are distributed directly from MyUtil.com.

1
2
3
4
5
6
7
8
9
10

sudo su
export PKG_PATH="http://myutil.com/ports/4.2/amd64/:ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.2/packages/amd64/"

# or, setenv PKG_PATH "http://myutil.com/ports/4.2/amd64/:ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.2/packages/amd64/"

pkg_add ruby-1.8.6.111 ruby-iconv-1.8.6.111 ruby-gems-1.0.1

# Perhaps Ruby-On-Rails:
gem install rails

1
2
3
4
5
6
7
8
9
10

sudo su
export PKG_PATH="http://myutil.com/ports/4.2/sparc64/:ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.2/packages/sparc64/"

# or, setenv PKG_PATH "http://myutil.com/ports/4.2/sparc64/:ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.2/packages/sparc64/"

pkg_add ruby-1.8.6.111 ruby-iconv-1.8.6.111 ruby-gems-1.0.1

# Maybe Rails?
gem install rails

Thanks, I find these useful and I hope you do too. I now do all my development from a OpenBSD (patched) Desktop and have made these binaries as secure as possible. All binaries are produced behind pf firewalls.

I have a old Mac Mini I can install macppc and produce binaries…if there is a demand.

MyUtil.com was hosted on a dedicated server from 1and1. Now I’m hosting on a dedicated server from m5hosting.com. 1and1 infrastructure was excellent when I leased these servers 3 years ago and I was one of their first customers. I can’t say that now as speed on my “100Mbps” connection to all my servers had dwindled to a paltry 180K/s to the west coast.

Then the the biggest problem was 1and1’s customer service. Sure they answer the phone. But then the person who answers the phone can only send one way messages to the techs. And the techs never send information back to the customer. Right before I dumped them, I took a perfectly healthy machine and started their re-image process to do speed tests with their OS instead of mine. 36 hours and many phone calls after my box disappeared, 1and1 customer service could only say “We will check with level 2 tier support. Check back with us in another 24 hours”. Boy though, once I used their cancellation website to dump them, they had me locked out of my prepaid servers within about an hour (my choice was cancel at end of contract, or, cancel in 30-120 days. BTW, MyUtil.com should have had 0 downtime…but I wasn’t expecting “end of contract” to mean 1and1 locks you out now and keeps your money.

I have been completely satisfied with m5hosting. It is very refreshing to have a support issue in the middle of the night, and the tech helping me designed the network. Plus now my sites scream!

I wanted to prototype my Gibberish translations before we have an actual translator. I grabbed gibberish_translate and started copying and pasting from Google Language Tools.

After five minutes of this, I thought there has got to be a better way. I googled for an API to the Google tools, and though I found none, I did find a scraping Ruby API called rtranslate. So…

gem install googletranslate

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  require 'rtranslate'
  def index
     # ...Mark's entire index method goes here unchanged
    if params[:filter] == "untranslated"
      count=0
      @paginated_keys.each do |key|  
        if ! @translated_messages[key]
          @translated_messages[key] = { 
          :to => Translate.t(@en_messages[key], Language::ENGLISH, session[:translation_locale] ),
          :from => @en_messages[key]
          }
        end
        break if (count += 1) == per_page
        sleep 1 # Let's be nice to google
      end
    end
  end
 # end of index from gibberish_translate's translations_controller.rb

And now Google does the work for me with the click of a mouse!

Note I did make some other changes to Mark’s code. There was a bug in translations_controller.rb in that it lost your current local when saving changes. To fix this I changed the set_translation_locale to use the session of there is no paramater:

1
2
3
4
5

  def set_translation_locale
    session[:translation_locale] = params[:translation_locale] if params[:translation_locale]
    session[:translation_locale] = Gibberish.languages.first if Gibberish.languages if ! session[:translation_locale]
  end

I also made some changes to gibberish_translate’s extractor.rb to handle Gibberish strings with default keys ("foo"[] is a valid Gibberish way of saying "foo"[:foo]):

1
2
3
4
5
6
7
8
9
10
11

    def message_pattern(start_token, end_token)
      /#{start_token}((?:[^#{end_token}](?:\\#{end_token})?)+)#{end_token}\[:*([a-z_]*)[,\]]/m
    end

    def add_messages(contents, start_token, end_token)
      contents.scan(message_pattern(start_token, end_token)).each do |text, key|
        key = text.tr('[  ]', '_').downcase if ( key == '' )
        add_message(key, remove_quotes(text, end_token))
      end
    end

The final tweaks I made was to make the find system call more portable (no -regex on OpenBSD) and also have it search for strings in my gibberish_rails plugin:

1
2
3
4
5
6
7
8

    def files_with_messages
      `find #{dirs_to_search.join(" ")} -type f '(' -name '*rb' -or -name '*.ml' ')'`.split.map(&:chomp)
    end

    def dirs_to_search
      %w(app config lib vendor/plugins/gibberish_rails).map { |dir| "#{RAILS_ROOT}/#{dir}" }
    end

Peace!

Portions of the above code Copyright© 2007 Peter Marklund

With migrating from Rails 1 to Rails 2, I have tried to simplify. When I wanted to prototype a multilingual Rails application I was very intrigued by Gibberish and it’s simplicity.

As all Gibberish does is translate strings and all this plugin attempts to do is translate srings in Rails. This plugin is in a very early prototype stage but I expect it to be useful none the less.

If you want full localization of dates, numbers, the world etc. check out some of the other more mature localization plugins.

If you are trying to localize your Rails strings with Gibberish, then this plugin is for you.

When I set out I didn’t even expect to make a plugin, just write some simple ruby in my project. However, it turns out there is a reason for the bloat in localization plugins…rails was never designed to be localized and has some quirks that lead to the necessity of overriding large core rails methods. The rails core team is obviously aware the problem and are working on a solution with ticket 9726. I’m hoping Rails ticket 9726 will make it to edge and then I’ll be able to simplify this plugin.

Without further adieu, I give you gibberish_rails.

Here is a link to the RDoc.

Quickie instructions (includes install for Gibberish).

./script/plugin install svn://errtheblog.com/svn/plugins/gibberish
./script/plugin install http://svn.myutil.com/projects/plugins/gibberish_rails/

Please read the README in it’s entirety before using.

Now you must translate your strings. I recommend using gibberish_translate. My next article will be on automatic prototyping translations with gibberish_translate and Google Language Tools.

While nightly backups and SVN are great, sometimes we make a mistake and rm something we need back. This article should also be useful to anyone who needs to do forensic analysis of a filesystem. This technique should also work under freebsd, netbsd, linux, solaris, dos, windows, etc. just the installation part of tools would be different and you might need a -t option to dls.

I did this yesterday on my development box under the /home partition. The first thing is to try to ensure nothing overwrites the deleted inodes.

In this example /home is /dev/wd0e. You can use df to determine your setup.

shutdown  now
# or, if you are not alone and want to give peope 2 minutes:   shutdown  +2

Now that we are in single user mode, unmount the disk

umount /home

Ok, the inodes are safe. Let’s undelete them. On OpenBSD we have, The Sleuth Kit an evolution of TCT and unrm is now called dls.

sudo su
export PKG_PATH="ftp://ftp2.usa.openbsd.org/pub/OpenBSD/4.2/packages/i386" 
# or, setenv PKG_PATH "ftp://ftp2.usa.openbsd.org/pub/OpenBSD/4.2/packages/i386" 
pkg_add sleuthkit

# Now, we will need tcl for the comeforth script referenced below.  Intall if you don't have:
pkg_add tcl-8.4.7p5
ln -s  /usr/local/bin/tclsh8.4 /usr/bin/tclsh

Now my /var has lots of free space. The following command will find all deleted inodes and place them in a file.

dls  /dev/wd0e > /var/tmp/undelete.bin

You can also yous fls on the raw device to report on directory information (file names). To get information on deleted sub directories, you would need to determine which inodes are directories and then use fls on those inodes.

If it is a text or source file you are looking for, you can just use something like less to search undelete.bin now.

Otherwise, if you want to extract files, I recommend comeforth. TCT does not build correctly on OpenBSD anymore. It is possible to get enough of TCT working for lazarus to run, but comeforth is much faster.

First, download and open comeforth. Now make it executable chmod +x ~face/comeforth/comeforth. We should already have tclsh installed above so now we run comeforth which is an interactive script. I am looking for Ruby-On-Rails files under app. So first I create a file of regular expressions that will match the output of the file command for the files I want to retrieve:

echo 'ASCII' > files
echo 'HTML' >> files

Script started on Mon Jan 14 15:13:05 2008
e5:/var/tmp> ~face/comeforth/comeforth

comeforth 1.12, Copyright (c) 2003-2004 Danamis Associates (http://danamis.com).
This program comes with ABSOLUTELY NO WARRANTY; this is free software, and you
are welcome to redistribute it under certain conditions; for details view the
GNU Lesser General Public License at http://www.gnu.org.

Data file: undelete.bin

Data block size: 4096

Recovery directory: recov

File type regex
([?] for help): < files

- Found 2 regexes to use in 'files'.

Block work dir: comeforth-5045.tmp

Progress indicator block interval: 24

Start at block: 1

Scanning data for matching blocks...
24, 0.01%, 1411.8 per sec, 2.9 min rem...

# Lots of output deleted ...

Finished scanning filesystem data in 0:09:26.

Inspect and assemble files? ([y]es/[q]uit): q

And thats it. You now have all your deleted files in recov. You can use grep to find the specific files you want. If the arglist is too long, then break it down with find. For example, lets say you where looking for the ruby class UsersController:

find . -print | xargs -L 10000 grep "^class UsersController"

You may have to delete some nulls at the end of your files. I tried dls -b but that didn’t seem to help.

I hope this is useful to someone else out there…

http://wiki.sleuthkit.org/index.php?title=Help_Documents http://wiki.sleuthkit.org/index.php?title=Tools_Using_TSK http://www.linuxhaxor.net/2007/12/26/undelete-files-in-linux-with-lazarus-and-unrm/

Thanks to Bernd Ahlers, the official OpenBSD ruby 1.8.6-111 ports are now in the OpenBSD CVS Repository.

I have updated my post and associated binaries Ruby 1.8.6p110 on OpenBSD 4.2 to use the official 111 port instead of mine.

OpenID makes sense. Dr. Nick’s multi-OpenIDs per user example app makes even more sense.

In the middle of integrating it into my project, gem-1.0.1 came out and broke ruby-openid-1.1.4. Dr. Nick’s great example no longer worked!

A little digging and I found Dr. Nick’s example uses the standard open_id_authentication. That has a patch to work on ruby-openid-2.0.2 and rails 2 which can be found here.

So in a nutshell, I grabbed openidauth_multiopenid-0.3.2 from Dr. Nick, removed a bunch of stuff from vendor plugins. Updated Rakefile, config/boot.rb, and config/environment.rb for rails 2.0.2. Patched vendor/plugins/open_id_authentication for ruby-openid-2.0.2. Regenerated db/migration/002_add_open_id_authentication_tables.rb. And installed ruby-openid-2.0.2 as a system gem.

As a little code is worth more than a thousand words, here is Dr. Nick’s example application fully ported to rails 2.0.2 in ZIP and TAR.gz.

For my port of Dr. Nick’s example above to work, you will need rails-2.0.2 and ruby-openid-2.0.2 installed as a gems.

http://drnicwilliams.com/2007/07/26/sample-app-rails-multiple-openids-per-user/
http://dev.rubyonrails.org/ticket/10604
http://openidenabled.com/ruby-openid/
http://svn.rubyonrails.org/rails/plugins/open_id_authentication/
http://openid.net/