PMA Media Group» Tools

Ruby file trimming app

hals — Fri, 17 Jul 2009 22:26:33 +0000

We recently had an interesting experience with very large files. These were comma delimited files (.csv) containing hundreds of thousands of records, each with a dozen or so fields.

e.g.

rec1,field2,,,,,,xxx,fieldn,,,1,2,3,,,fieldx

rec2,field22,,,a,s,d,fieldmore,,,,etc

recn,field2n,,,,ring,,,,ring,1,2,,,hello?,,etc

While testing the setup, we had smaller files to work with. The goal was to create a new file containing only the first field from each record.

e.g.

rec1

rec2

recn

During testing this was easily done by opening the file in a spreadsheet program (such as OpenOffice), which would split the records on the comma delimiter and place each field in a different column. Then, it was easy to select the first column and write it out to the new file.

On switching to production files, we discovered that OpenOffice has a limit of 65k rows – a fraction of what we needed. We then tried some other spreadsheet programs, which produced the same results. We knew there was at least one spreadsheet program that would work, but it was not open source.

At this point the comment was made: “well, we ARE ruby developers …”

And that lead to the following simple solution to the problem at hand.

With a few lines of ruby code, the source files could be read in, line by line, split on the comma delimiter, and the first entry written out to the destination file.

So, when the usual tools just don’t work – remember that a new ruby tool might be just around the corner.

#!/usr/bin/ruby

# trimfile.rb

require “rubygems”

require “ruby-debug”

class Trimfile

attr_accessor :fileName, :newFile

def initialize(fileName, newFile)

puts “\nSplit off first comma delimited item of each line.”

@fnam = fileName

if @fnam == nil then @fnam = “trimin.txt” end

@newfnam = newFile

if @newfnam == nil then @newfnam = “trimout.txt” end

linecount = 0

puts “\nFilenames – input: #{@fnam}, output: #{@newfnam}”

aFile = File.new(@newfnam, “w”)

IO.foreach(@fnam) do |line|

aFile.puts line.split(‘,’)[0]

linecount += 1

end

aFile.close

puts “\nTotal lines: #{linecount}”

end

test = Trimfile.new(ARGV[0], ARGV[1])

Working with Git

Narshlob — Fri, 19 Jun 2009 23:34:48 +0000

This tutorial covers all the commands (hopefully) we’ll need for the projects we build here at PMA. If there’s anything that needs to be added to it, feel free to comment.

Starting with the basics, we’ll first cover retrieving a project:

git clone [repository]

This will get the currently active branch from the repository

Obviously you’ll want to do something with this newly retrieved working copy of the project. Let’s first create a branch for the new features/bug fixes we’ll be coding.

git checkout -b [newbranchname] origin

Ok, so we got a new branch. While we’re coding, it’s a good idea to commit tons of times to preserve the changes we’ve just made. Don’t worry about the log, we’ll make it pretty later. Just commit often.

git add [filename(s)]
git commit

If it’s a small change that doesn’t require much explanation, you can use these commands

git add *
git commit -m "The commit message"

Or, even shorter

git commit -a -m "The commit message"

You finished that feature so now it’s time to merge that branch with the master branch (or some other branch depending on what VCS (Version Control System) paradigm you/your team chose). First, you should make those hard-to-read commits less hard-to-read. Let’s rebase!
Please note that if you rebase after pushing to the repository, you will create problems for those pulling from that repository. Rebase changes the history of the project. Your teammates merges will not be fast-forward[able]. It won’t be pretty, trust me.

git rebase -i HEAD~[number of commits back]

You’ll now be looking at something similar to this:

pick ce86448 A random commit
pick a8564a9 Another random commit

How you order things in this editor will affect the order of the commits. Note that merge commits are not shown here. They aren’t editable.
Replacing “pick” with “edit” will allow you to edit the changes you made as well as the commit message.
After you’ve edited the files you wanted to edit, you can now

git add *

then

git commit --amend

and move on.

As I mentioned before, you have the opportunity to clean up the mess you made with all those many commits using the rebase option. Here’s how:

Run the
```
git rebase -i HEAD~[x]
```
command from earlier

Replace “pick” with “squash” on the commit you want to be combined with one exactly previous to it.

pick ace72dd I squashed these commits. I'm cool.
squash e99fd59 This commit will be sqaushed with the one above it
pick d0770e8 commited again
pick af845d0 I'm really committed

Pretty straight forward and easy. Everyone loves rebasing
If you want to know more about git-rebase, I recommend checking it out here.

Now, you’ve made all these changes and everything looks great. What are you gonna call this pretty new feature? Are you gonna tag it? I would..

git tag -a [fancy_feature] -m "A fancy message for a fancy feature"

If you happen to leave out the -m, git will open an editor and you’ll be able to add your fancy message there, just like with commit!
Read more about tags here.
One scenario for using tags could be that, within a project, one wants to keep track of versions. Each commit could be tagged with a version number like with bug fixes in some VCSs. If something goes wrong, it’s really simple to go back to a previous version using git rebase, as we’ve already shown.

You’ve squashed those ugly commits, changed the commit message(s), and tagged everything. Time to merge. Switch to the master branch (or whatever branch you’re wanting to merge with) and type this command which will merge [branchname] with the current branch:

git merge [branchname]

You’ll probably want to fix any conflicts and continue with the merge.
It’s now safe to delete [branchname] because all the changes from that branch are now on the current one.

git branch -d [branchname]

This next feature is pretty neat. Say you’ve done a bunch of changes that haven’t been committed yet and you realize you aren’t on a feature branch. Here’s what you do:

git stash
git stash branch [newbranchname]

This will stash away all uncommitted changes, create a new branch and check it out, then unstash all your stashed changes into that new branch. Awesome

Stash is also useful in scenarios where you don’t want to commit yet but you need to switch to a different branch and do something. You could stash the current changes using the above git stash command, do the needed changes, then switch back to the branch you were working on and use this command to unstash the changes:

git stash apply

It’s a good idea to check out the other things offered by git stash (git-stash)

To get this new branch into the origin repository, do:

git push origin [branchname]

To delete a branch from the origin repository, do:

git push origin :[branchname]

Don’t forget the colon!

Another scenario; your co-worker does some work on a feature and gets stuck. You don’t want to type on their computer cause yours is set up just the way you want it. Is there a solution to this quandary? Yeah. There is…
Tell them to push their changes then do this:

git fetch origin [remotebranchname]:[localbranchname]

You now have the branch they were working on locally and can modify to your hearts content.

Noteworthy Notes

There’s a difference between

git pull

and

git fetch

The difference is that “git pull” will run a “git fetch” then a “git merge” to merge the retrieved head into the current branch.

git log -p path/to/file.rb

This command will show the history of a specific file;

git blame path/to/file.rb

will go line by line in a file and give a short description + the name of the person that changed the line last (brilliant, actually)

From the Git manual

check it out
A few configuration variables (see git-config(1)) can make it easy to push both branches to your public tree. (See the section called “Setting up a public repository”.)

$ cat &gt;&gt; .git/config &lt;
      [remote "mytree"]
           url =  master.kernel.org:/pub/scm/linux/kernel/git/aegl/linux-2.6.git
           push = release
           push = test
EOF

Then you can push both the test and release trees using git-push(1):

git push mytree

or push just one of the test and release branches using:

git push mytree test

git push mytree release

To rebase your current working tree to obtain the changes from the master tree,
Suppose that you create a branch “mywork” on a remote-tracking branch “origin”, and create some commits on top of it:

git checkout -b mywork origin
$vi file.txt
$ git commit
$ vi otherfile.txt
$ git commit

…
You have performed no merges into mywork, so it is just a simple linear sequence of patches on top of “origin”:
o–o–o <-- origin
\
o--o--o <-- mywork
Some more interesting work has been done in the upstream project, and "origin" has advanced:
o--o--O--o--o--o <-- origin
\
a--b--c <-- mywork
At this point, you could use "pull" to merge your changes back in; the result would create a new merge commit, like this:
o--o--O--o--o--o <-- origin
\ \
a--b--c--m <-- mywork
However, if you prefer to keep the history in mywork a simple series of commits without any merges, you may instead choose to use git-rebase(1):

$ git checkout mywork
$ git rebase origin

This will remove each of your commits from mywork, temporarily saving them as patches (in a directory named “.git/rebase-apply”), update mywork to point at the latest version of origin, then apply each of the saved patches to the new mywork. The result will look like:
o–o–O–o–o–o <-- origin
\
a'--b'--c' <-- mywork
In the process, it may discover conflicts. In that case it will stop and allow you to fix the conflicts; after fixing conflicts, use git add to update the index with those contents, and then, instead of running git commit, just run

$ git rebase --continue

and git will continue applying the rest of the patches.
At any point you may use the —abort option to abort this process and return mywork to the state it had before you started the rebase:

The commands

Here’s a list of all commands covered in this tutorial:
git-clone
git-checkout
git-add
git-commit
git-rebase
git-tag
git-merge
git-branch
git-stash
git-push
git-pull
git-fetch

Culerity and Celerity, JavaScript enabled testing in Cucumber

fugufish — Mon, 08 Jun 2009 23:02:30 +0000

As you may know, testing JavaScript inesnsive applications using Cucumber can be a pain. There are several ways to handle this; Selenium, Watir and the like. The easiest way I have found however is by using Celerity a jRuby API for htmlUnit, a fully functional headless browser that completely supports JavaScript, and you don’t have to serve your application on jRuby to use it!. I was able to get Celerity to play nice duruing Cucumber tests by doing the following: First, you will need of course jRuby. The Culerity gem will start up Celerity in the jRuby environment, and proxy the actual browser through it. Download jRuby and extract it to wherever you want it located. In my case I put it in /opt, and set your PATH accordingly.

export PATH=$HOME/jruby/bin:$PATH

Next install the Celerity gem in jRuby (we are using the github version rather than the ruby forge version, as it provides a later version):

jruby -S gem install jarib-celerity --source=http://gems.github.com

Now install the Culerity gem. This provides the interface between Celerity and your environment without forcing you to run in jRuby

 gem install langalex-culerity --source http://gems.github.com

Place this in your test environment:

# config/environments/test.rb
 
config.gem "langalex-culerity", :lib =&gt; false

I call :lib => false here to avoid loading the gem. This is done in the file generated by culerity. Finally remove features/steps/webrat_steps.rb as it will conflict with Culerity, and run:

script/generate culerity

Enjoy! Culerity should have very similar syntax to webrat, however keep in mind you may see some differences that you may need to adjust.

Google Chrome Commercial

Brandon Buttars — Fri, 08 May 2009 20:08:48 +0000

Here is a sweet Google Chrome advertisement from YouTube. I wonder when Microsoft will come out with one for IE8. Ah oh. Here goes another Mac vs. Windows like fight.

Tutorial: How to Set up Ruby on Rails and MySQL on Mac OSX Leopard

Chuck Wood — Tue, 07 Apr 2009 20:07:20 +0000

Mac OSX has become extremely popular for Ruby on Rails development. Part of this is the inclusion of Ruby as part of the operating system on Mac OSX Leopard. Here is a quick tutorial on how to get Ruby on Rails and MySQL set up on your Mac OSX Leopard machine. In order to complete this tutorial, you need the Xcode tools installed. You can get them off of the applications disk that came with your mac or from http://developer.apple.com/TOOLS/xcode/.

Once you have Xcode Tools installed, open your terminal (in Applications > Utilities) and let’s get started!

1. Update RubyGems.

sudo gem update --system

2. Install Ruby on Rails.

sudo gem install rails

3. Download the MySQL (x86) package from this page.

4. When the .dmg file loads, install both packages (.pkg files) and the MySQL.prefPane.

5. Once MySQL is installed, install the mysql gem. (Thanks wonko.com for the tip.)

sudo env ARCHFLAGS="-arch i386" gem install mysql -- \
  --with-mysql-dir=/usr/local/mysql --with-mysql-lib=/usr/local/mysql/lib \
  --with-mysql-include=/usr/local/mysql/include

That’s it! You’re ready to go! Now all you have to do is create a rails application and you’re ready to go.

Skype Beta 2.8 – Screen Sharing

Brandon Buttars — Fri, 03 Apr 2009 17:05:27 +0000

I’ve always been a SKYPE fan and a user even though there seem to be a few haters out there. Most of the haters are haters cause they’ve had a bad internet connection when trying to use Skype. Overall if you have a good connection it doesn’t get much clearer and better than SKYPE. SKYPE is cheap, has some great plugins and functionality, and I’m sure a whole lot more uses that I even use it for.

Well last night I was doing some browsing and trying to find some good screencasting software to use that was FREE of course. In my searching I ran across SKYPE and had to check it out. I went to the SKYPE Beta 2.8 download and downloaded it to check it out. I’ve got to say Skype Screensharing is pretty much the bomb. The problem it currently has as of this post is that it doesn’t display your cursor, but it does all the actions of the cursor at an incredible speed. I was not disappointed. Once the cursor shows up it will be unstoppable.

So SKYPE haters stop hating and give it a try. If your internet connection sucks, don’t push the sucky blame onto SKYPE, really give credit where it’s due.

Usage Ideas

Here are some ideas that I thought may motivate some of you to give it a try based on some practical business and freelancer use:

Technical Support Screencasting – this is pretty self explanitory, whether it’s the teaching the senior citizen’s how to copy and paste or teach your dad how to check his email this would work great.
Application Training Screencasting – being able to train someone how to use your application so easily without an encrypted link and long url to have to send them, SKYPE makes this process very seemless. Just double click on your contact and select to share screen.
Website Training Screencasting – say you have a client and you need to show them how to manage their new site you have just created for them. Turn on your screen recorder and the SKYPE Screen Sharing and now you have live training with a recorded video to upload to YouTube or some other video sharing website (recommend making the video private for obvious reasons).
Webinars - I think SKYPE has a 25 person limit on conference calls, but I’m assuming that you could share your screen with all 25 people within a conference call. I’ll have to double check on that one.
Online Education – teach online courses through Skype now using a combination of video and screensharing.

Things to keep in mind about the SKYPE Beta 2.8 is that it is for a Mac. Yet another reason to buy a Mac. I’m sure it will come out on Windows eventually potentially post Beta release on Mac, but it doesn’t matter the viewer. I tested it last night sharing my screen from my MacBook Pro to my HP Windows Vista machine and it worked perfect.

If you need anymore convincing then you are one hard cookie to sell. Please leave any comments or feedback you might have. Feel free to share this post so we can get more feedback. Oh yeah and did I mention, I LOVE SKYPE!

P.S. Skype is starting to show up on iPhones and I even have it installed on my Google G1 Android phone. They’re still kind of early in development but don’t be surprised if you can make calls from your mobile phones using your mobile internet without burning up your minutes.

What? NetBeans 6.5.1 Can Now Recognize My Cucumber .feature Files!

Chris Gunnels — Thu, 02 Apr 2009 22:59:18 +0000

After watching Ryan Bates’ screencast on Cucumber, I wanted to start creating my code first through BDD. I used Netbeans 6.5.1 to follow Ryan’s screencast and I found that NetBeans doesn’t recognize the .feature file that you create when you run a Cucumber test. I searched Google for about 3 hours for a plugin, but no such luck! I couldn’t believe it, I thought somebody had to of created a cukes plugin by now. Well, I decided to help the world and of course, myself, by creating a NetBeans plugin that will recognize the .feature files. Here’s a quick view of a .feature file after the plugin has been installed:

I have uploaded the plugin to the NetBeans Plugin site, but you can download it from this link: NetBeans Cucumber Plugin -v 1.0

EDIT: Netbeans has been down for a week or two so we’ve made the download available here: Netbeans Cucumber Plugin
After you download the project navigate to: CucumberFileType / build / org-myorg-cucumberfiletype.nbm
Install the .nbm file with netbeans and your ready to go.
END EDIT

To install the plugin all you have to do is:

Download it (Duh)
Open NetBeans and go to Tools -> Plugins
In the Plugins dialog click on the “Downloaded” tab
Click “Add Plugins…”
Browse to where ever you downloaded the .nbm file and click “Open”
Last, click the “Install” button and you finished.

Hope you like it. Please report any bugs or changes.

Comment, Tweet, Appreciate!

20 Mac Apps You Should Install

Brandon Buttars — Wed, 01 Apr 2009 20:17:35 +0000

I know this list isn’t the same as everyone elses. I know we all have our preferences. I’m sure some of our other designers would have Dreamweaver on their list and some of the Programmers would have Net Beans, but this is my list. Leave you list in the comments below. I’ll make this short and sweet and just give a little bit of information about the different programs. I recommend going to their websites and learning even more of how they can help you. Download the FREE trials and take them for a spin.

The list is in alphabetical order just because that was easy.

PMA Media Group» Tools

Ruby file trimming app

Working with Git

Noteworthy Notes

From the Git manual

The commands

Culerity and Celerity, JavaScript enabled testing in Cucumber

Google Chrome Commercial

Tutorial: How to Set up Ruby on Rails and MySQL on Mac OSX Leopard

Skype Beta 2.8 – Screen Sharing

Usage Ideas

What? NetBeans 6.5.1 Can Now Recognize My Cucumber .feature Files!

20 Mac Apps You Should Install

Adium

Adobe Fireworks

Adobe Illustrator

Adobe Photoshop

Firefox

Flickr Uploader

Fluid App

Google Notifier

iTunes

Little Snapper

MindNode

NeoOffice

On The Job

PandoraBoy

Quicksilver

ScreenFlow

Sequel Pro

Skype

TextMate

Tweetr