Jonathan Gnagy's Blog

Rails for everyone!

Make it Rake

I’ve heard people say great things Apache Ant and old faithful make, but I haven’t heard enough people talking about how wonderful rake is to work with. I’ve used rake for many simple and complex tasks, from importing YAML files into a Rails project, to managing LDAP users. Rake offers the full power of Ruby, but still offers a fairly simple DSL.

Here’s an excellent railscast about rake: http://railscasts.com/episodes/66-custom-rake-tasks.

Here’s some code that demonstrates creating a simple rake task:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
task :mkdirs do
  # Let's create some directories
  ["lib", "conf", "src"].each do |dir|
    Dir.mkdirs(dir) unless File.directory?(dir)
  end
end

# This task requires the 'mkdirs' task
task :copy_source => :mkdirs do
  require 'ftools'
  # Copy the source into the src directory...
  ["main.rb", "classes.rb"].each do |file|
    File.copy("/path/to/sources/#{file}", "src/#{file}")
  end
end

These two simple tasks create some directories and copy some source files. It also demonstrates task dependencies and pure Ruby file operations.

Move pretty much complete

I’ve successfully unloaded the truck I rented (with the help of a couple good friends). My back hurts, I cut my hand pretty deep, and I’ve got a seemingly never ending list of groceries to buy from the closest store.

That drive seemed a lot longer than I remember it being. Maybe it was the truck, who knows. After all that driving, and all that unloading, I’m still not done. Now I need to unpack all these boxes too, and rebuild my dismantled furniture. I will admit though, that it’s nice to feel like the hard part of the move is over. Now I have to wait a couple weeks for my wife and baby girl to fly up here to live with me. Maybe the hard part isn’t over just yet.

Ruby Syslog Server

I uploaded a very rough draft of a syslog server written in Ruby. Here’s a link to it. I talked about it in a previous blog post, and I think now its ready to show the world. Here’s a quick breakdown of some of the key features:

  • Uses ActiveRecord for DB backend
  • Heavily Multithreaded
    • Makes extensive use of Queues and Mutexes
    • Works in Ruby 1.9, JRuby, and Mostly in MacRuby for Native threads. (Obviously) Uses green threads in Ruby 1.8
    • Utilizes Thread Pools to maintain a set number of Threads
  • Listens for UDP, TCP, or both.
  • Uses a YAML config file
  • Fully documented with RDoc
  • Performs exceptionally under heavy load. I tested 10,000+ simultaneous connections with the included test client (under /test/).
  • Written with MVC in mind. (I say “in mind” because there really is no “view” portion to this app)

That said, its still pretty new, and I’m still working on it. I welcome any comments about the code or any bugs encountered. Keep in mind that this SHOULD NOT BE USED IN PRODUCTION. I offer no warranty, implied or otherwise. I license this code under the GPLv3, even if I forgot to include a copy of the license in the tarball.

All-in-all, its actually a pretty simple app. Not that much actual code involved; just a little cleverness on how to process things one piece at a time. Here’s some of my favorite code, along with some parts that help it make more sense:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# ...
        # This methods tells the server to pull all logs from the LogQueue and 'process' them.
        # Essentially, this moves logs from the incoming LogQueue to a process ThreadPool for
        # controlled processing. This method prevents an insurge of messages from eating up all
        # the server's CPU by limiting the number of logs processed at a time. After the log is
        # parsed in the process pool, it is queued up for storage in the storage ThreadPool.
        def process_logs
                until @log_queue.empty?
                        @queue_flushed = false if @queue_flushed
                        log = pop_log
                        @process_pool.add_job {
                                puts "Processing Log: #{log.object_id.to_s(16)}"
                                parsed_log = @parser.parse log
                                @storage_pool.add_job {@storage.store_log parsed_log}
                        }
                end
                if !@queue_flushed
                        puts "Log Queue Flush Complete @ #{Time.now}" if !logs_queued?
                        @queue_flushed = true
                end
        end
        
        # Utilizes the LogMutex to synchronize initial storage of log packets in the LogQueue
        def enqueue_log(log_data)
                @mutex.synchronize { @log_queue.push log_data }
        end
        
        # Utilizes the LogMutex to synchronize removing log packets from the LogQueue
        def pop_log
                return @mutex.synchronize { @log_queue.pop }
        end
        
        # Utilizes the LogMutex to synchronize checking how many logs are currently queued in the
        # LogQueue.
        def logs_queued?
                return @mutex.synchronize { @log_queue.size > 0 }
        end
# ...

Hopefully, with more work, this could have some serious potential.

Another Picture of Marissa

Here’s a picture of Marissa showing off which team she is hoping will win. I’m like her in that I’m not really a big fan of the Saints, so we’ll both be cheering with the Cardinals win.

Marissa Likes the Cardinals

If you want to see a bigger version of the image, check out the Media Section of my blog.

Even More Blog Improvements

I’ve had some time to burn since I’m not working for a couple weeks while I’m on paternity leave. During the night, my wife and I take shifts watching Marissa sleep (yeah, we’re paranoid like that). During my last shift (this one), I’ve accomplished quite a bit. I’ve managed to:

  • Utilize Google’s Translate javascript API to provide site-wide translations
  • Create a fully-functional, albeit simple, Todo / Checklist system and integrate into my blog
    • Uses Scriptaculous effects
    • I serialized the list items into an Array, so I only need a single column for an arbitrary number of list items. Also, no need for a separate “ListItems” model / table.
    • It allows me to mark lists as public, so everyone can see them. By default lists are private.
    • All adding and removing from the list is done via Ajax (with a non-javascript HTTP/1.0 POST fall-through), so it looks pretty snazzy.
    • I’ve started an experimental option to apply my tagging system to lists as well, but I haven’t figured out how I will make it all work, or how to display it.
  • Polish up some sections of my site that weren’t quite passing XHTML 1.0 Strict compliance tests. I’m sure there are more out there, so feel free to comment if you happen to come upon a page this isn’t. Come to think of it, I’m sure the list stuff isn’t yet, since I didn’t check it.
  • Make more of the site RESTful and support more pure XML operations (for scripting and such).

I’m sure I did more stuff, but I can’t really remember it right now. My eyes are awfully blurry… I’ll try to figure out some code to post tomorrow.

New Blog Features

I added some minor improvements to my blog. Some of these feature include:

  • All tags now offer feeds, even if they aren’t in the “Popular Tags”. Just click on the tag, and the grey banner contains a feed link.
  • All successful searches (i.e., those that return results) allow for subscriptions via Atom feeds. Similar to tag feeds, they exist in the grey banner that details the search term.
  • All uploaded media is available and searchable from the newly added “Media” link under the also newly added “Extras” section on the right.
  • Several small improvements to admin areas…

Here’s some code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def feeds
    case params[:feed]
      when "all"
        @entries = Entry.find(
          :all, 
          :limit => 25, 
          :order => "updated_at DESC"
        )
      when "by_tag"
        @entries = Tag.find_by_name(params[:tag]).entries.find(
          :all, 
          :limit => 25, 
          :order => 'updated_at DESC'
        )
      when "by_search"
        query = "%#{params[:q]}%"
        @entries = Entry.find(
          :all, 
          :conditions => ["body LIKE ?", query], 
          :limit => 25, 
          :order => 'updated_at DESC'
        )
    end # case params[:feed]
    respond_to do |format|
      format.atom  { render :layout => nil }
    end
end

A More Interesting Blog Post

My wife gave birth to my daughter, Marissa Wendy Gnagy, yesterday (12/22/2009) at about 4AM. I can’t believe how much of a life changing experience it would be. I tried to mentally prepare myself, but I wasn’t even close. I’m just ecstatic. Here’s a picture I took while holding her (luckily we got her to sleep for a little while). She’s my little angel.

Marissa

Recursive Ruby meets Fibonacci

I was trying out recursive programming in Ruby the other day. I figured a good way to try it out was using a pretty well known mathematical sequence called the Fibonacci sequence. In math, the sequence is defined as

Fn = Fn – 1 + Fn – 2

with seed values of

F0 = 0 and F1 = 1

so its a prime candidate for a recursive algorithm in Ruby. I started off with a working, but very slow, method that looks like this:

1
2
3
4
5
6
7
8
9
def fib(n)
  if (n == 0)
    return 0 
  elsif (n == 1)
    return 1
  else
    return fib(n - 2) + fib(n - 1)
  end
end

For the first few numbers in the sequence this method performs well enough, but as the number gets larger, things start slowing way down. For example:

1
2
3
4
5
fib 6  #=> 8
fib 9  #=> 34
fib 20 #=> 6765
fib 30 #=> 832040 (took about 5 seconds to complete)
fib 40 #=> ? (waited for over a minute with CPU pegged...)

So, obviously this stuff is working, but its not really very practical. Next I took a stab at using an array to create an in-memory cache, which sped things up immensely:

1
2
3
4
5
6
7
8
9
10
11
12
13
@@cache = [] # this is an in-memory cache
def fib_cached(n)
  if @@cache[n]
    return @@cache[n] # return the cached value, if it exists
  else
    if (n == 0) || (n == 1)
      @@cache[n] = n
    else
      @@cache[n] = fib_cached(n - 2) + fib_cached(n - 1)
    end
    return @@cache[n] # always put the value in cache, then return it
  end
end

This speed difference was unbelievable:

1
2
3
4
5
6
7
fib_cached 6   #=> 8
fib_cached 9   #=> 34
fib_cached 20  #=> 6765
fib_cached 30  #=> 832040
fib_cached 40  #=> 102334155
fib_cached 99  #=> 218922995834555169026
fib_cached 300 #=> 222232244629420445529739893461909967206666939096499764990979600

I tried different numbers up to 999 and wasn’t able slow it down at all, which is great.

Finally, I found a shortcut on a blog (which I can’t remember the URL for), that uses the Golden Ratio (phi) to very quickly find numbers in the Fibonacci sequence. It works very well, but I think since I’m not really using Ruby’s more advanced math modules, the numbers skew a little (then a lot) as the sequence number gets large.

1
2
3
4
5
6
module Math
  Phi = (Math.sqrt(5) + 1) / 2
end
def fib_by_phi(n)
  (((Math::Phi ** n) - ((1 - Math::Phi) ** n) ) / Math.sqrt(5)).to_i
end

Although this was extremely fast, it didn’t quite produce accurate results at higher sequence numbers:

1
2
3
4
5
6
7
fib_by_phi 6   #=> 8
fib_by_phi 9   #=> 34
fib_by_phi 20  #=> 6765
fib_by_phi 30  #=> 832040
fib_by_phi 40  #=> 102334155
fib_by_phi 99  #=> 218922995834555891712
fib_by_phi 300 #=> 222232244629422676106398124069050176556246085874450435841982464

In the end, I found that the cached recursive algorithm was the best balance of speed and reliability, trading off memory for CPU cycles. Here’s some code that you can use to test things and get a very precise measurement of performance:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
def stop_watch(&block)
  start_time = Time.now
  puts "Started at #{start_time}"
  begin
    yield
  rescue Exception => e
    puts "Block died: #{e}"
  end
  end_time = Time.now
  puts "Ended at #{end_time}"
  puts "Finished in #{end_time - start_time} seconds."
end
#run each method in a stop_watch
stop_watch do
  puts fib_cached 9999
end
stop_watch do
  puts fib_by_phi 9999
end

Baby is Almost Here

I’m very excited that my baby is almost here. I know its going to be very difficult in almost every way imaginable, but I know it will all be worth it. I can’t wait to tell my baby everything I know. At first, I wasn’t really sure if I was ready to be a father yet, but I’ve come around to the idea, and I’m going to give it my best shot. I think with the support of my wife and the rest of my family, I’ll do just fine.

Ruby and Numbers

Sometimes you need to work with numbers in Ruby. We all do it, and its necessary. But sometimes you run across a situation where you need to take a string that looks like a number, and convert it to a number. For instance, let say we’ve got the following:


number = "3"

Now lets say that I want to use this in some math… It won’t work very well:


wrong_answer = number * 14 # => TypeError: String can't be coerced into Fixnum

So, how can we make this work? Take a look at this:


answer = Integer(number) * 14 # => 42

This isn’t the only method that provides this kind of functionality. Need to work with Floats? Try this:

1
2
number = "3.14"
answer = Float(number) * (4**2) # => 50.24

Just thought I’d share some useful Ruby code. I haven’t posted code in a while, and I figured it might help somebody out there.