Google App Engine, JRuby, Sinatra and some fun!

Yesterday I was experimenting with Google App Engine for a little project that I was working on. I literally started from ground zero and could do my thing after a long night. I'm blogging about it to share you the idea and save you some time googling solutions. So, here is the thing in brief: I wanted to parse some feed on periodical basis and send an email with new entries.

Looks like a 10 minutes with Nokogiri and cron jobs. Actually that's true except of the fact that I need to pay for a VPS so that I can run the script with various gems (since I needed to do some work on the feeds before sending, but that's for another topic) and for using cron jobs. Thus, I decided to try something new this time and I went with GAE since it has memecached that I can use as an LRU cache, also it has cronjobs and finally it has a mail system. I'm using JRuby and Sinatra for this project.

Here are the steps:

Install JRuby, I'm using RVM on my machine:

rvm install jruby-1.5.5
rvm use jruby

Install needed gems, those versions are the ones which worked for me:

gem install jbundle
gem install appengine-sdk -v "1.4.0"
gem install google-appengine -v "0.0.19" 

Create a simple app:

 
appcfg.rb generate_app notifier
cd notifier 
 

Now let's create the following files (Please note that the following code can be further enhanced):

Gemfile

 
# Critical default settings:
disable_system_gems
disable_rubygems
bundle_path ".gems/bundler_gems"

# List gems to bundle here:
gem "appengine-apis"
gem "appengine-rack"
gem "sinatra"
gem "sinatra-reloader"
gem "jruby-rack", "1.0.4"
gem "nokogiri","1.5.0.beta.3"

config.ru

(fill the sender/receiver emails and the feed URL)

 
require 'sinatra' 

set :sender_email, 'SENDER_EMAIL'
set :receiver_email, 'RCEIVER_EMAIL'
set :feed_url, 'FEED_URL'

require 'app'
run Sinatra::Application

app.rb

 
require 'digest/sha1'
require 'appengine-apis/memcache'
require 'appengine-apis/logger'
require 'appengine-apis/mail'
require 'appengine-apis/urlfetch'
require 'sinatra'
require 'sinatra/reloader' if development?
require 'nokogiri'


class Fetcher 
  
  def initialize(sender_email,receiver_email,feed_url)
    @sender_email = sender_email
    @receiver_email = receiver_email
    @feed_url = feed_url
  end

  def act
    send_email(create_message(get_results))
    "done"
  end
  
  private 
  def memcache
    @memcached ||= AppEngine::Memcache.new(:namespace=>"cache")
  end

        # get the new results only
  def filter_results(results)
    res = []
    results.each do |hashed_guid,info|
      if memcache.get(hashed_guid).nil?
        memcache.set(hashed_guid, 1)
        res << info
      end
    end
    res
  end

        # parse the feed
  def get_results
    results = {}
    doc = Nokogiri::XML(AppEngine::URLFetch.fetch(@feed_url).body)
    doc.css('item').each do |item|
      title = item.at_css('title').content
      link = item.at_css('link').content
      description = item.at_css('description').content
      guid = item.at_css('guid').content
      results[Digest::SHA1.hexdigest(guid)] = {:title => title,:description => description, :link => link}
    end
    filter_results(results)
  end

        # create email message
  def create_message(results)
    template = ""
    results.each do |info|
      template += <<-DOC
      
        Title: #{info[:title]}
        Description: #{info[:description]}
        Link: #{info[:link]}
        --------------------------------------------------------------------------------------------
      DOC
    end
    template
  end


  def send_email(message)
    AppEngine::Mail::send(@sender_email,@receiver_email,"New updates for the site",message) if message.length > 0
  end
end


get '/' do
  "Hello World!"
end


get '/notify' do
  Fetcher.new(settings.sender_email,settings.receiver_email,settings.feed_url).act
end

cron.xml

(You can set the period here, I'm using 15 minutes intervals)

<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
  <cron>
    <url>/notify</url>
    <description>fetch new data</description>
    <schedule>every 15 minutes</schedule>
  </cron>
</cronentries>

Now, create an application-id at appspot.com. Then go to Administration >> Permissions and make sure the sender/receiver emails play some role in the app. Personally, I granted them the developer role.

Now, go back to the source and modify the application-id in WEB-INF/app.yaml

application your-app-id

We should be ready now, start development server, watch the console and make sure no exceptions are there:

dev_appserver.rb .

If everything is OK, go to http://localhost:8080/notfiy and you should see "Done".

Let's go live now:

(the first time you execute this, you will prompted to submit the GAE email/pass combination)

appcfg.rb update .

It should now be running at http://your-app-id.appspot.com, and in few minutes you should be receiving the first email. If not, go to app engine page then to Main >> Logs and check what's the problem.

I like this stack cause it's Zero cent cost, Zero deployment time, flexible, and it gives me exactly the freedom I want in terms of needed gems. Give it a shot, you won't regret it!