Configuration for mailjet email delivery with Ruby on Rails
Posted by Wes in 
Setting up mailjet.com to deliver your mail via Ruby on Rails? Here’s how, because their Getting Started is nothing but placeholder headers right now.
git: merge a single commit
Sometimes, you have one commit you want to get into production, but it’s located after other changes that you’re not ready to merge in yet. How can you get that single git commit into a different branch?
First, you have to know the SHA of the commit you want:
git checkout branch-with-commit-on-it git log
Highlight and copy the SHA of the commit you want to grab.
git checkout master # -n => don't commit, just merge changes so we can review and commit ourself git cherry-pick -n [The commit’s SHA-1 Hash] # review git diff –cached # commit if all is well git commit -a -m “merge SHA1 ..."
If you’re feeling confident, you can skip the -n and merge the single commit in directly and save a minute.
Note, this isn’t a merge, so it’s possible you could have some conflicts down the road when you merge the original commit into this branch. You’re creating a brand new commit object.
Ruby’s ParseExcel and the Extra Date
We have a feature in one of our application which allows the user to upload data from an Excel spreadsheet. One of the columns read in is a date field. Because we’re dealing with end users, sometimes those dates come in as “Date” objects, sometimes they come in as “String”s, and sometimes they come in as a number.
When Excel saves a date as a numeric, it is that value’s number of days since January 0, 1900 (No, we can’t say Dec 31, 1899). So a spreadsheet would store “May 1, 2011″ as 40664 if it was formatted as a number.
That’s great, peachy even, but if you’ll notice, 40664 days from Dec 31, 1899 is May 2, 2011, not May 1st. What happened? Compatibility.
When Microsoft introduced Excel way back when, Lotus123 dominated the spreadsheet marking. In order for Microsoft to be able to compete, it had to be compatible with Lotus123. It had to be compatible with all the formulas, all the features, and all the bugs. One of those bugs had to do with an erroneous leap day in 1900. This means that when you read in numbers from a spreadsheet in order to translate them to a date, you have to account for that extra leap day.
The solution? Just subtract one from the numeric. It might look something like this (from the Rails console):
Date.civil(1899, 12, 31) + 40664.days - 1.day
We could have did that anyway and just chalked it up to something weird, but now, at least, we know why it’s weird.
Notes:
- OpenOffice is also fully compatible with that bug.
- Dates and Times in Excel
Gateway timeout with nginx/passenger standalone
I needed to run Rails apps with both ruby 1.8.7 and 1.9.2 on the same server.
Passenger Standalone to the rescue! Setting up my 1.9.2 app as a standalone server and setting it up as proxy to it worked great.
Until we had to upload and process some files. Turns out, the gateway server would timeout, even though the process was still processing on the app server.
The proxing nginx server would reply with “Gateway Timeout 504″.
The fix from the nginx documentation: proxy_read_timeout
My proxy config after this:
location / {
proxy_pass http://127.0.0.1:3010;
proxy_read_timeout 240s;
}
Git: Commit Early, Commit Often
Don’t Be Afraid of Commitment
The phrase “save early, save often” was a mantra of early computer science. It’s good advice, but it’s only half the story. If you’ve ever sat down for a fast and furious coding session only to realize hours later that you removed something important, you know the frustration of not being able to get it back. It can mean hours lost.
Getting in the habit of regular commits has a number of benefits. First, you can go back to any previously committed version if your coding goes off-track. You can reference earlier parts of your work even if you don’t need to revert to them. Best of all, it an actually have a positive impact on your code itself.
Better Code through Committing
Just as Test-Driven Development influences us to write a larger number of shorter/simpler methods, frequent committing pushes us to think of atomic changes. Atomic in this context means the smallest possible self-sufficient change. Consider this snippet from git log:
commit b0e77532b5a8cf236d95f1b3324aabc194568c60 Author: Wally Date: Tue Feb 29 23:58:05 2011 -0600 added comments to blog posts
This is an example of a commit done at the end of a feature. Lots of code has probably gone into this, and if you wanted to see any of the steps along the way, you’re out of luck. It’s also not very easy to see what was changed in the app.
An Example of Frequent Commits
Now look at this version:
commit e8bba8ad1a4b5c1353c328b505bfa2a9f4816d07 Author: Alice Date: Tue Feb 29 14:58:05 2011 -0600 added edit/delete links to each comment if user has permissions commit 8bba8ad1a4b5c1353c328b505bfa2a9f4816d07e Author: Alice Date: Tue Feb 29 13:58:05 2011 -0600 allowed admins to edit/update/delete any comments commit bba8ad1a4b5c1353c328b505bfa2a9f4816d07e8 Author: Alice Date: Tue Feb 29 11:58:05 2011 -0600 restricted edit/update/delete actions to user's own comments commit ba8ad1a4b5c1353c328b505bfa2a9f4816d07e8b Author: Alice Date: Tue Feb 29 11:36:20 2011 -0600 created controller/views commit a8ad1a4b5c1353c328b505bfa2a9f4816d07e8bb Author: Alice Date: Tue Feb 29 10:03:31 2011 -0600 made comments nestable in the model commit 8ad1a4b5c1353c328b505bfa2a9f4816d07e8bba Author: Alice Date: Tue Feb 29 9:39:24 2011 -0600 defined activerecord associations among users, posts, and comments commit ad1a4b5c1353c328b505bfa2a9f4816d07e8bba8 Author: Alice Date: Tue Feb 29 08:38:01 2011 -0600 added base comment model and migrations
You can clearly see the evolution of this feature, and it makes sense. Not only can you step back (if user permissions were implemented incorrectly, for instance) but Alice was “forced” to think about each logical step of her development process, instead of jumping in head first. It’s engineering 101 – break a problem in its atomic elements, and attack each of those.
Incidentally, this can also help explain to coworkers, bosses, and clients why a seemingly simple task took longer than expected. As you squirm and attempt to justify what you know is an elegant solution, you might not remember all the steps that got you there.
Git remembers.
Thebes, profiling spork, backup rubygem
Thebes, a new minimal sphinx gem for Rails
Thebes is a wrapper around Sphinx, the search engine we use on most of our projects. Thebes differs from other solutions by staying as far away from your Rails code as possible. Instead of hiding the Sphinx configuration file behind a domain-specific language, this library assumes you will write Sphinx config files by hand. In Thebes, you edit an ERB template of your Sphinx configuration and populate it with variables at generation time. For developers needing the most flexible or fastest solution possible, this is a great way to work with Sphinx.
We’ve been doing some interesting things with search and reporting that are going to require faster lookups than directly querying the database. From the article “the [Thinking Sphinx project] has a lot of complexity and ties to ActiveRecord 2.x code. Consequently, the porting of TS to Rails 3 isn’t turning out to be the smooth road we hoped for. So, for Rails 3 projects, this looks like a good way to go if you’re willing to get your hands dirty and build some sphinx files yourself.
Profiling Spork for faster start-up time
Spork allows you to preload Rails environment files into a process, then it forks that process and runs your tests against the new process. In essence, your tests will start faster because they’re not loading everything. You can specify files you want to be reloaded each time (for instance, model files).
The code:
This prints out everything being loaded up, so you can move files that don’t change into the preload block for that extra bit of snappiness.
Backup, a rubygem for database and file backups
Backup is a RubyGem (for UNIX-like operating systems: Linux, Mac OSX) that allows you to configure and perform backups in a simple manner using an elegant Ruby DSL. It supports various databases (MySQL, PostgreSQL, MongoDB and Redis), it supports various storage locations (Amazon S3, Rackspace Cloud Files, Dropbox, any remote server through FTP, SFTP, SCP and RSync), it can archive files and folders, it can cycle backups, it can do incremental backups, it can compress backups, it can encrypt backups (OpenSSL or GPG), it can notify you about successful and/or failed backups (Mail or Twitter). It is very extensible and easy to add new functionality to. It’s easy to use.
Check out the README for all the details, but this allows you to backup your app via command line, pushing to S3 or rsync’ing to another server. You can schedule it with the fantastic Whenever gem too.
A :limit of Rails’ Migrations
Migrations in Ruby on Rails use the “:limit” symbol to set the maximum length of the underlying field’s data type. Take for example, the following example migration:
create_table :things do |t| t.string :name, :limit => 32 t.string :description t.timestamps end
By default, Rails will create :description as data type “varchar(255)” and :name as “varchar(32)” in a MySQL database. But did you know you can set :limit to be greater than 255?
For whatever reason, many of us have gained the impression that 255 is the longest :string can be, but that just isn’t the case. If I wanted the :description field in the example above to be greater than 255, I could just define it as follows:
t.string :description, :limit => 1024
In fact, strings (i.e. varchars) in MySQL can hold up to 65,535 bytes of data.
The opinionated nature of Ruby on Rails is a great asset in most instances, but we have to be careful not to let its opinions :limit us.
Note: I’m pretty sure Rails sets the default limit of strings to be 255 for two reasons: 1) cross database compatability, and 2) MySQL’s InnoDB (utf-8) engine can’t index varchar fields exceeding 255 characters.
Further Reading
Git Branching After the Fact
We have a pretty simple Git work flow here at Databasically: Just work in master. There are a couple reasons for this: we’re not a large team, and we have a very rapid (i.e. daily) release cycle. I had been used to creating branches for every new feature, so when I found out we primarily work on the “master” branch, I was a bit shocked.
I know what you’re thinking, “But what if you go down a path and realize you need to branch in order to put in a ‘hotfix’, or you find out the story is more involved than initially thought?”: you just create a branch after the fact. Here’s how to do it in just a few easy steps:
Step 1: Know where you are and where you want to go
The first thing you need to know is where you are and where you want to go.
In the case (completely contrived example) above, I need to add a fix to what is in production (SHA: fa87cd9). So I want to roll everything back and start working from that hash point.
Step 2: Create a new branch
Now that we know where we want to go, we need to first create a new branch. We create a branch in order to force Git to remember our current line of work. In some ways, you can think of Git branches as inodes in Unix/Linux: as long as a file descriptor [branch] is pointing to an inode, it can’t be fully deleted [reset]. To do this, we’re just going to issue the “git branch <branch name>” command.
I should note that before you do this, make sure you’re clean. “Add” and “commit”, or “stash”, what you’re currently working on.
So here, I created the branch “hotfix” and you can see that HEAD, master, and hotfix are all pointing to the same SHA. You can make sure your branch exists, by issuing the “git branch” command. Now, we can make our changes in master and push to production.
Step 3: There is No … step 3
This means you, Bruce.
Step 4: Reset Master
Now that we have a branch (created in step 2) we’ll reset HEAD to our chosen hash. We do that using git’s “reset” command:
git reset --hard fa87cd9
The log will look something like this:
After making our change and committing, our log looks like this:
Now we can push that into production and get back to what we were working on.
Step 5: Rebase and Merge
Step 5a: Rebase Master
In order to get back to what we were working on, we need to first rebase the changes we made in master into production. To do that, first checkout hotfix and run git rebase master
git checkout hotfix git rebase master
By rebasing master, we replay those changes made in master onto our “hotfix” branch. Our hotfix branch should now look like this:
Notice where HEAD and hotfix are, and notice also where master is. It is now safe to merge everything back to master – and without leaving those unsightly branch paths.
Step 5b: Merge
First things first: check out master. Next: merge hotfix
git checkout master git merge hotfix
You output will look something like this:
And your log will look like this:
And everything is caught up. At this point you can delete the hotfix branch:
git branch -d hotfix
Conclusion
This model works well for us, but I’m sure as we grow we’ll likely have to adopt more involved models such as the “A successful Git branching model” used by nvie.com. As we grow and modify our processes, we’ll be sure to let you know what we find to work and what we find which doesn’t.
Further Reading
Notes:
In this post, I have been calling “git log” with my alias “git lol” here is the details of the alias:
log --graph --decorate --date=local --pretty=format:'%h %cd (%an) %s%d'
default_scope: Use Cases, Caveats, and Work Arounds
In Ruby on Rails, named scopes are class methods used to restrict and organize the data searched for. In SQL terms, a named scope adds to the conditional (WHERE) and sorting (ORDER) sections of a query. See Railscast #108 for more information.
In Rails 2.3.x, a new type of scoping was added to the API: the default_scope. By using “default_scope”, one could restrict the data retrieved by every query without the need for extra method calls; it would just happen by “default”. If you wanted to have your data sorted in a particular manner, you could add “default_scope, :order => ‘created_at DESC’” to your model. From then on, all data retrieved would be ordered by “created_at” in a “descending” manner.
Caveats
In general, default_scope should be avoided if possible. Here are a few reasons:
- Out of site, out of mind: Because you don’t see that you are scoping your data as you query it, it’s easy to forget that it is in actuality being filtered. This can lead to a lot of head scratching until you remember the default_scope.
- default_scope is inherited: All subclasses of the original model will inherit the default scoping. This may not be the behavior you desire.
- Extra overhead: It’s one thing if you need your data sorted every single time, it’s quite another if you don’t. By using default_scope, you may be unnecessarily burdening your database.
Use Cases
Like curry, default_scope is not inherently evil (obscure Phineas and Ferb reference), and there are instances where using it makes good sense. As an example, here at Databasically, we use default_scope to limit data retrieved by one application to a subset of what is available in a table. There will never be an instance where all the available data will be required, and so we limit it by default.
Here are the two use cases:
- When only a subset of the data is ever required
- When the data must be returned in a specific order every time
Work Arounds
I highlighted the words “only” and “every” in the section on use cases to make a point: default_scope should be used with caution. The fact is, however, that it’s not an “only” and “every” world, and as such, we need work arounds. In our case, it’s by using “with_exclusive_scope” and the undocumented “unscoped” (Rails 3.x only)
with_exclusive_scope example:
Article.with_exclusive_scope { find(:all) }
unscoped example (Rails 3.x only):
Article.unscoped
Conclusion
In general, we like to keep our code as DRY as possible, but in cases like default_scope, we prefer to be more explicit. We recognize the value of default_scope, but only use it when we absolutely need to, and even then we try to think of alternative methods.
How about you? We would really like to hear from you about your experiences with default scopings. What considerations do you take into account when choosing whether or not to use default_scope?
Further Reading
What time is it? Or, handling timezones in Rails.
As a followup to a stack overflow answer, I thought I would give some examples of working with time zones in rails.
What does Rails timezone support do for me?
- Stores everything in UTC in the database
- Allows you to set an application default timezone and/or timezones for your users
- Automatically converts UTC in the database to the correct zone and back
What zones are available?
You can get a list of timezones with rake tasks:
# Displays names of all time zones recognized by the Rails TimeZone class, grouped by offset. rake time:zones:all # Displays names of time zones recognized by the Rails TimeZone class with the same offset as the system local time rake time:zones:local # Displays names of US time zones recognized by the Rails TimeZone class, grouped by offset. rake time:zones:us
Setting the default time zone
In your environment.rb (Rails 2) or application.rb (Rails 3) file, you can set the default timezone:
config.time_zone = 'Central Time (US & Canada)'
What does this do? By setting an application-wide timezone, any datetime will be stored in UTC in the database, but will be translated when we access it.
Set a timezone for a user
You can use time_zone_select to get a list of timezones for a user to pick from. The third argument is a list of “priority” zones that will appear first.
time_zone_select( "user", 'time_zone', TimeZone.us_zones, :default => "Pacific Time (US & Canada)")
Once the value is saved in the database, you’ll want to set it for each request, per user:
before_filter :set_timezone
def set_timezone
# current_user.time_zone #=> 'Central Time (US & Canada)'
Time.zone = current_user.time_zone || 'Central Time (US & Canada)'
end
[UPDATE: You'll need a field 'time_zone' in your user table!]
[UPDATE: You probably want to stay DRY and refer to the configured value instead of specifying the timezone value in both places:
Time.zone = current_user.time_zone || MyAppName::Application.config.time_zone]
Let me know if you have questions or improvements and I’ll integrate them into the article. Thanks!