Large CSV Import on Heroku

In the recent months, I’ve been lucky to work with Tyler Tringas, the founder of Storemapper.co. For your info, store mapper is a store locator snippet that you can put on your website and shopify page. We already have a lot of happy customer such as Beardbrand and Uber. If you are a business owner looking for a store locator solution, try it out!

One interesting that we solved recently is how to import a large CSV file on heroku. Tyler has documented it well in his blog post. If you have any question regarding that, feel free to comment here ūüôā

Books: Agile Web Development with Rails 4

I just finished this book from The Pragmatic Bookshelf. Agile Web Development with Rails 4

agile web development with rails

The book is divided into three major part. The first part is an introductory part about ruby, how to install Rails, etc. The second part discussed about how to build an online catalog using Rails. You can check out the online catalog that I built from this book here. I wish the look is not so bland, but anyway it does what it is supposed to do. The last part discussed many Rails topics in-depth, such as Active Records, Rails dependencies, etc.

If you have finished your first introductory materials to Rails and looking for more learning materials, I can recommend this book for you.

Setting Environment Variable with Figaro Gem

When developing with Rails, often we have to include confidential information in our code. For example if we need to input our API key or some login information. If you are working on open source software or you share your repository with other people, you might want to remove this confidential information from your codes.

One method to secure the information is to move them to environment variables, and read the information variable from the code. Something like this

Pusher.key    = ENV["pusher_key"]

However, setting the environment variable manually might become annoyance if you have to do it many times.

Figaro gem to the rescue! With Figaro, you can just place all your variables in a file named config/application.yml, for example like this

pusher_key: "7381a978f7dd7fas2437"

The previous code will read the environment variable from the application.yml file. Of course you have to remove the application.yml from your repository.

You also can set environment specific variables using figaro. For example:

pusher_key: "7381a978f7dd7fas2437"
test:
  pusher_key: "ad612fh4a44dcac1fb28"

If you want to learn more about environment variable and rails, this article by Taylor Mock and Daniel Tahoe might be useful.

Rails 4.2 Launched with Some Cool Feature

A couple of days ago, Rails 4.2 is officially launched. There are a couple of feature that I know I will be glad for. One thing is adequate records that make simple query like ‘find’ is faster. Aaron Patterson’s blog cover the details of adequate records.

Other thing that will be useful is the active job. Active job is an adapter for various queuing system like Resque, Sidekiq, Sucker Punch, Delayed Job, etc. This will come handy, for example when it’s time to graduate from Sucker Punch into something more sophisticated like sidekiq, I don’t have to rewrite the job.

Web console is also interesting, though some of my friends said that this make the current Rails bloated. I find this feature is fun. See the gem page for some demos.

You can see the complete features list in the release notes.

Just to celebrate, I upgraded brentimana.com to Rails 4.2. By following the upgrading guide, I manage the upgrades without many glitches. In brentimana’s case, I just had to watch out about the responders, that now has been extracted into the responders gem. Voila, brentimana is now running on Rails 4.2

Brentimana on Rails 4.2
Brentimana on Rails 4.2

Actually, the new Rails has many small but useful “hidden” features. Justin Weiss cover them nicely here.

Skip bundle exec when using rbenv

If you are using rbenv and bundler like I did, you must be feeling tired writing ‘bundle exec blabla’ everytime you use executables in your rails project. Fortunately, you don’t have to. You can use this rbenv plugin to automatically bundle exec your command. By default this plugin ignore ‘gem’ command, since normally you shouldn’t run gem with bundle exec.

While we are at it, another useful rbenv plugin is rbenv-gem-rehash. That plugin automatically run ‘rbenv rehash’ everytime you install a new gem. So, you don’t need to rehash everytime you run ‘bundle install’

This is a short post, but hopefully it’s usefull enough. See you!

Single Process Asynchronous Job with Sucker Punch

In a previous post we discussed about running asynchronous job with sidekiq. Sidekiq will spawn it’s workers processes and look for jobs on queue. But what if we don’t want to use separate processes to do asynchronous job?

Enter sucker punch. ¬†Sucker punch enable us to do asynchronous processing within a single job. ¬†This will reduce memory usage in your hosting and reduce hosting cost in Heroku. I use it often when I develop a mockup on Heroku free hosting.It won’t be wrong to call it a poor men’s sidekiq.

Sucker punch is simple to use. I just want to remind you to take special precaution when interacting with active record in an asynchronous process. Use ActiveRecord::Base.connection_pool.with_connection to not exhaust the connection in the ppol, as per the example in the readme:

# app/jobs/awesome_job.rb

class AwesomeJob
  include SuckerPunch::Job

  def perform(user_id)
    ActiveRecord::Base.connection_pool.with_connection do
      user = User.find(user_id)
      user.update_attributes(is_awesome: true)
    end
  end
end

I also like to give some delay before starting an async job.

class Job
  include SuckerPunch::Job

  def perform(data)
    puts data
  end

  def later(sec, data)
    after(sec) { perform(data) }
  end
end

Job.new.async.perform("asdf")
Job.new.async.later(60, "asdf") # `perform` will be excuted 60 sec. later

So far sucker punch is my best friend when working with Heroku, since I’m a cheapskate, hahah.

Scheduled Task with Sidekiq and Sidetiq

Sometimes, we need to do scheduled task for our Rails app, for example for caching, sending mass notification, etc. There are multiple solution available, one of them is by using Sidekiq and Sidetiq.

Sidekiq is a gem that enable us to do asynchronous works by creating background workers. What makes it better from other solution like Resque is Sidekiq uses multithreading so it is much more memory efficient than resque. For queuing, Sidekiq requires Redis, so make sure to setup it first.

Now, if you want to make recurring scheduled jobs, we can add Sidetiq on top of Sidekiq. With Sidetiq, you can easily define the interval of the schedule like daily, minutely, etc. Please use ruby > 2.0 when using Sidetiq, since it has a known memory issues when used with ruby 1.9.3 or lower.  You also need to use Redis > 2.6, so check your redis version out. It costs me half a day just to figure that out, hahah.

If you do everything correctly, Sidetiq can give you this beautiful UI:

sidetiq ui

The recurring jobs are only listed in production environment though. This is because in Rails development environment, eager loading is disabled by default.

Ok, hope that helps you in using Sidekiq and Sidetiq. Cheers!

A Web Developer Journal