Remove all those .DS_Store files
Posted by Wes in 
I don’t like committing .DS_Store files into my projects, so here’s an easy way to remove them:
Run this from your project folder:
# Recursively erase all .DS_Store files in this folder and below
find . -name \.DS_Store -exec rm -v {} \;
Also, you can prevent the DS_Store files from being created. Run this from the Terminal:
defaults write com.apple.desktopservices DSDontWriteNetworkStores true
git: merge a single commit
Sometimes, you have one commit you want to get into production, but it’s located after other changes that you’re not ready to merge in yet. How can you get that single git commit into a different branch?
First, you have to know the SHA of the commit you want:
git checkout branch-with-commit-on-it git log
Highlight and copy the SHA of the commit you want to grab.
git checkout master # -n => don't commit, just merge changes so we can review and commit ourself git cherry-pick -n [The commit’s SHA-1 Hash] # review git diff –cached # commit if all is well git commit -a -m “merge SHA1 ..."
If you’re feeling confident, you can skip the -n and merge the single commit in directly and save a minute.
Note, this isn’t a merge, so it’s possible you could have some conflicts down the road when you merge the original commit into this branch. You’re creating a brand new commit object.
Mac OS X: find the program running on a port
Sometimes, there’s a program running on a port and you don’t know what it is. How do you find out?
Converting mysql databases to UTF8
UTF8 is the way to go when you’re creating a new database for an application, but how do you get your existing applications upgraded?
Tools we love
We spend a lot of time on our computers, so we spend a lot of time trying out new tools. Here are some of our favorites:
Backblaze
Backblaze backs your computer up, offsite, automatically.
Killer features:
- Protect from your computer crashing
- Restore a file you accidentally delete or change
- Offsite storage
- Unlimited space
- Can encrypt your data if you need/want to
- Locate My Computer, if your computer is lost or stolen, backblaze can help you track it down
Cost:
- $5 per month per computer
- $50 per year per computer
Dropbox
Dropbox synchronizes files across multiple computers.
Killer features:
- Keep folders in synch across multiple computers. No more emailing files to yourself
- Share folders with other people. For instance, I shared a folder with my bookkeeper to put receipts into, we have a shared folder for work projects, you can share folders with clients to transfer files back and forth
- Public folder. If you want to send a big file to someone and can’t email it, you can put the file in your dropbox public folder. That uploads it and you can send a link to the person and they can download it from the dropbox server.
- Mini backup. Files in here are available from the dropbox website, so in a pinch you can login to the Dropbox website and get that presentation you forgot.
Cost:
- Free, can store up to 2GB
- $10/month for 50GB storage
- $20/month for 100GB storage
Ruby’s ParseExcel and the Extra Date
We have a feature in one of our application which allows the user to upload data from an Excel spreadsheet. One of the columns read in is a date field. Because we’re dealing with end users, sometimes those dates come in as “Date” objects, sometimes they come in as “String”s, and sometimes they come in as a number.
When Excel saves a date as a numeric, it is that value’s number of days since January 0, 1900 (No, we can’t say Dec 31, 1899). So a spreadsheet would store “May 1, 2011″ as 40664 if it was formatted as a number.
That’s great, peachy even, but if you’ll notice, 40664 days from Dec 31, 1899 is May 2, 2011, not May 1st. What happened? Compatibility.
When Microsoft introduced Excel way back when, Lotus123 dominated the spreadsheet marking. In order for Microsoft to be able to compete, it had to be compatible with Lotus123. It had to be compatible with all the formulas, all the features, and all the bugs. One of those bugs had to do with an erroneous leap day in 1900. This means that when you read in numbers from a spreadsheet in order to translate them to a date, you have to account for that extra leap day.
The solution? Just subtract one from the numeric. It might look something like this (from the Rails console):
Date.civil(1899, 12, 31) + 40664.days - 1.day
We could have did that anyway and just chalked it up to something weird, but now, at least, we know why it’s weird.
Notes:
- OpenOffice is also fully compatible with that bug.
- Dates and Times in Excel
Gateway timeout with nginx/passenger standalone
I needed to run Rails apps with both ruby 1.8.7 and 1.9.2 on the same server.
Passenger Standalone to the rescue! Setting up my 1.9.2 app as a standalone server and setting it up as proxy to it worked great.
Until we had to upload and process some files. Turns out, the gateway server would timeout, even though the process was still processing on the app server.
The proxing nginx server would reply with “Gateway Timeout 504″.
The fix from the nginx documentation: proxy_read_timeout
My proxy config after this:
location / {
proxy_pass http://127.0.0.1:3010;
proxy_read_timeout 240s;
}
Git: Commit Early, Commit Often
Don’t Be Afraid of Commitment
The phrase “save early, save often” was a mantra of early computer science. It’s good advice, but it’s only half the story. If you’ve ever sat down for a fast and furious coding session only to realize hours later that you removed something important, you know the frustration of not being able to get it back. It can mean hours lost.
Getting in the habit of regular commits has a number of benefits. First, you can go back to any previously committed version if your coding goes off-track. You can reference earlier parts of your work even if you don’t need to revert to them. Best of all, it an actually have a positive impact on your code itself.
Better Code through Committing
Just as Test-Driven Development influences us to write a larger number of shorter/simpler methods, frequent committing pushes us to think of atomic changes. Atomic in this context means the smallest possible self-sufficient change. Consider this snippet from git log:
commit b0e77532b5a8cf236d95f1b3324aabc194568c60 Author: Wally Date: Tue Feb 29 23:58:05 2011 -0600 added comments to blog posts
This is an example of a commit done at the end of a feature. Lots of code has probably gone into this, and if you wanted to see any of the steps along the way, you’re out of luck. It’s also not very easy to see what was changed in the app.
An Example of Frequent Commits
Now look at this version:
commit e8bba8ad1a4b5c1353c328b505bfa2a9f4816d07 Author: Alice Date: Tue Feb 29 14:58:05 2011 -0600 added edit/delete links to each comment if user has permissions commit 8bba8ad1a4b5c1353c328b505bfa2a9f4816d07e Author: Alice Date: Tue Feb 29 13:58:05 2011 -0600 allowed admins to edit/update/delete any comments commit bba8ad1a4b5c1353c328b505bfa2a9f4816d07e8 Author: Alice Date: Tue Feb 29 11:58:05 2011 -0600 restricted edit/update/delete actions to user's own comments commit ba8ad1a4b5c1353c328b505bfa2a9f4816d07e8b Author: Alice Date: Tue Feb 29 11:36:20 2011 -0600 created controller/views commit a8ad1a4b5c1353c328b505bfa2a9f4816d07e8bb Author: Alice Date: Tue Feb 29 10:03:31 2011 -0600 made comments nestable in the model commit 8ad1a4b5c1353c328b505bfa2a9f4816d07e8bba Author: Alice Date: Tue Feb 29 9:39:24 2011 -0600 defined activerecord associations among users, posts, and comments commit ad1a4b5c1353c328b505bfa2a9f4816d07e8bba8 Author: Alice Date: Tue Feb 29 08:38:01 2011 -0600 added base comment model and migrations
You can clearly see the evolution of this feature, and it makes sense. Not only can you step back (if user permissions were implemented incorrectly, for instance) but Alice was “forced” to think about each logical step of her development process, instead of jumping in head first. It’s engineering 101 – break a problem in its atomic elements, and attack each of those.
Incidentally, this can also help explain to coworkers, bosses, and clients why a seemingly simple task took longer than expected. As you squirm and attempt to justify what you know is an elegant solution, you might not remember all the steps that got you there.
Git remembers.
Thebes, profiling spork, backup rubygem
Thebes, a new minimal sphinx gem for Rails
Thebes is a wrapper around Sphinx, the search engine we use on most of our projects. Thebes differs from other solutions by staying as far away from your Rails code as possible. Instead of hiding the Sphinx configuration file behind a domain-specific language, this library assumes you will write Sphinx config files by hand. In Thebes, you edit an ERB template of your Sphinx configuration and populate it with variables at generation time. For developers needing the most flexible or fastest solution possible, this is a great way to work with Sphinx.
We’ve been doing some interesting things with search and reporting that are going to require faster lookups than directly querying the database. From the article “the [Thinking Sphinx project] has a lot of complexity and ties to ActiveRecord 2.x code. Consequently, the porting of TS to Rails 3 isn’t turning out to be the smooth road we hoped for. So, for Rails 3 projects, this looks like a good way to go if you’re willing to get your hands dirty and build some sphinx files yourself.
Profiling Spork for faster start-up time
Spork allows you to preload Rails environment files into a process, then it forks that process and runs your tests against the new process. In essence, your tests will start faster because they’re not loading everything. You can specify files you want to be reloaded each time (for instance, model files).
The code:
This prints out everything being loaded up, so you can move files that don’t change into the preload block for that extra bit of snappiness.
Backup, a rubygem for database and file backups
Backup is a RubyGem (for UNIX-like operating systems: Linux, Mac OSX) that allows you to configure and perform backups in a simple manner using an elegant Ruby DSL. It supports various databases (MySQL, PostgreSQL, MongoDB and Redis), it supports various storage locations (Amazon S3, Rackspace Cloud Files, Dropbox, any remote server through FTP, SFTP, SCP and RSync), it can archive files and folders, it can cycle backups, it can do incremental backups, it can compress backups, it can encrypt backups (OpenSSL or GPG), it can notify you about successful and/or failed backups (Mail or Twitter). It is very extensible and easy to add new functionality to. It’s easy to use.
Check out the README for all the details, but this allows you to backup your app via command line, pushing to S3 or rsync’ing to another server. You can schedule it with the fantastic Whenever gem too.
A :limit of Rails’ Migrations
Migrations in Ruby on Rails use the “:limit” symbol to set the maximum length of the underlying field’s data type. Take for example, the following example migration:
create_table :things do |t| t.string :name, :limit => 32 t.string :description t.timestamps end
By default, Rails will create :description as data type “varchar(255)” and :name as “varchar(32)” in a MySQL database. But did you know you can set :limit to be greater than 255?
For whatever reason, many of us have gained the impression that 255 is the longest :string can be, but that just isn’t the case. If I wanted the :description field in the example above to be greater than 255, I could just define it as follows:
t.string :description, :limit => 1024
In fact, strings (i.e. varchars) in MySQL can hold up to 65,535 bytes of data.
The opinionated nature of Ruby on Rails is a great asset in most instances, but we have to be careful not to let its opinions :limit us.
Note: I’m pretty sure Rails sets the default limit of strings to be 255 for two reasons: 1) cross database compatability, and 2) MySQL’s InnoDB (utf-8) engine can’t index varchar fields exceeding 255 characters.