Book: Pragmatic Thinking and Learning
The frisson never died. The excitement kept me turning one page after another. Andy Hunt’s Pragmatic Thinking and learning : Refactor Your Wetware is life changing and I highly recommend it to all Software Developers new and old.
Pragmatic Thinking and Learning (PTL) is full of ideas and insights. It is based on a lot of research and background reading. It would save you a lot of trouble of going through a lot of books and research to learn the same material which might not be written this well. I had already read some of the books mentioned in the book, some were on my candidate list and some I added to my candidate list based on citations from this book but this is a concise collection of all these tips and tricks you learn from all these books (some of them I will never read as they are research on neuroscience and nursing professionals).
What does Pragmatic Programmer and Pragmatic Thinking and Learning have in common besides practical advice, tons of insight and being my all time favorite books? Andy Hunt. Andy is awesome. Not only he writes well he is a very articulate speaker. In fact the way I got introduced to this book while it was being written was a talk he gave at my company. This book is so much more than that talk I loved.
I will write more about the practical tips as I act on them in the future in this blog or on my twitter but for now I just wanted to give a shout out for this fantastic book.
If you want to buy it from my favorite book seller online here is the link.
*nix: Comparing two files
Everybody knows about diff.
I read about sdiff today for side by side diff.
Here is the one I found useful in some cases but I forget it so I am saving it here:
NAME
cmp – compare two files
SYNOPSIS
cmp [-l | -s] file1 file2 [skip1 [skip2]]
DESCRIPTION
The cmp utility compares two files of any type and writes the results to
the standard output. By default, cmp is silent if the files are the
same; if they differ, the byte and line number at which the first differ-
ence occurred is reported.
May be I will add some examples to show the difference but later.
Software Development: Correctness, Completeness, Performance, In That Order
Software development is the act of converting thoughts into actions. In other words goal of software development is to convert requirements into code but this can’t be done in one step and in a single iteration.
Each iteration in the software process you need to get a piece of requirement convert it into design, write a test, add new code and potentially refactor existing code to implement it.
For every piece in each iteration you want to follow the ordered sequence of correctness, completeness and performance. In other words, get it right, get it working, get it fast.
When understanding the requirement you want to make sure you are getting the right requirement basically understand what customers want not what they think they want. Often in requirement gathering phase the customer is biased to what they are used to or how they want something rather than letting you know the input and output they desire from a feature. Your job is to get to the bottom of it by asking good questions.
In the context of requirements completeness means that you not only understand the input and output but the entire context to implement the feature correctly. Once you understand what and in what context only then you turn your attention to performance requirements (if any) as discussion of performance earlier might not be in context and well understood where the customer is coming from. All these discussion do not have to happen at the same time but in should happen in that order.
I think I have already explained the hardest part in the whole iteration. Requirements is often where we get it wrong and the hardest/most-expensive one to correct after the fact.
When it comes to writing code you also follow this sequence, make sure you are implementing what you intend on implementing by writing unit tests first. The act of trying to write test first often force you to think about what you are about to do. You repeat this process until you have met all the requirements for the feature (completeness) and only then you measure performance and do any performance tuning if necessary. Software developers are not only often guilty of the thought crime of premature optimization but actually commit this crime if not stopped in time.
Do you think my thoughts are incorrect, incomplete and would not scale? Please send me a thought patch in the comments.
Javascript: My first Mozilla Ubiquity Command
I learned about Mozilla Ubiquity yesterday through my colleague Arnab Deka and I read the article about on the bus on my way back home on Instapaper iPhone application and I installed it a little over an hour ago and I am already loving it.
It already comes a lot of command built in but when I looked for a dictionary look-up command I didn’t find it. The closed was the define command which takes you to answer.com.
I thought this was a good way to add my favorite dictionary.com look up command and familiarize myself with how to add a new ubiquity command.
Here is what I came up with in less than 15 minutes using the template and tutorial:
/* This is Ali Rizvi's ubiquity first command */
CmdUtils.CreateCommand({
name: "dictionary",
// icon: "http://saaridev.wordpress.com",
homepage: "http://saaridev.wordpress.com",
author: { name: "Ali Rizvi", email: "@gmail.com"},
license: "Ruby License",
description: "Command to lookup a work on dictioanry.com",
help: "dictionary <word-to-lookup>",
takes: {"input": noun_arb_text},
preview: function( pblock, input ) {
searchText = jQuery.trim(input.text);
if(searchText.length < 1) {
pblock.innerHTML = "Searches for word on dictionary.com";
return;
}
var previewTemplate = "Searches dictionary.com for <b>${query}</b>";
var previewData = {query: searchText};
pblock.innerHTML = CmdUtils.renderTemplate(previewTemplate, previewData);
},
execute: function(input) {
var url = "http://www.dictionary.com/browse/{QUERY}";
var query = input.text ;
var urlString = url.replace("{QUERY}", query);
Utils.openUrlInBrowser(urlString);
}
});
When I hit the save button ubiquity command editor posted this gist. Anybody know how to add js syntax highlighting on github gists?
Vim: Auto indenting based on filetype
I had two different behaviors on my windows and linux machine when I used Vim to edit my ruby files.
I liked the auto-indent behavior I had on my windows machine but I did not know how to make it happen on my other linux machine.
I finally invested sometime to find out the difference between the two:
At first I thought I was simply missing
set autoindent
but I verified that I had that in both of my vimrc files.
The difference was that on my windows machine I was using the vimrc_example file which came with this line:
" load indent files, to automatically do language-dependent indenting. filetype plugin indent on
This did the trick and I am so happy now.
Now when I start an if block and hit enter the second line starts with the appropriate indentation and when I type end the editor automatically indents it to the previous indentation level of the appropriate block of code.
Cheers!
Java: Rediscovering the basics
It’s been a while since I did some Java but now it seems to be the right tool for the job at hand so I am rediscovering Java with a masochistic twist, ‘look boss no IDEs’.
I am experimenting with the SSJ Java library for Stochastic Simulation as I mentioned in an earlier post and here is some sample source code:
import umontreal.iro.lecuyer.randvar.*;
import umontreal.iro.lecuyer.probdist.*;
import umontreal.iro.lecuyer.rng.*;
public class GenerateDemandArrivals {
private RandomStream randomStream;
private RandomVariateGen expArrival;
private RandomVariateGen poissonArrival;
private double lambda;
private int times;
public GenerateDemandArrivals() {
randomStream = new MRG32k3a();
lambda = 10.0;
times = 5;
expArrival = new RandomVariateGen(randomStream, new ExponentialDist(lambda));
poissonArrival = new PoissonGen(randomStream, new PoissonDist(lambda));
}
public void demandArrival() {
for (int i = 0; i < times; i++) {
System.out.println("Here is the next random int : " + randomStream.nextInt(0, 10));
}
}
public void demandArrivalExpDist() {
for (int i = 0; i < times; i++) {
System.out.println("Here is the next exponential dist double : " + expArrival.nextDouble());
}
}
public void demandArrivalPoissonDist() {
for (int i = 0; i < times; i++) {
System.out.println("Here is the next poisson dist double : " + poissonArrival.nextDouble());
}
}
public static void main(String[] arg) {
new GenerateDemandArrivals().demandArrival();
new GenerateDemandArrivals().demandArrivalExpDist();
new GenerateDemandArrivals().demandArrivalPoissonDist();
}
}
I was tried of doing the following everytime I changed the source code in my favorite editor Vim:
javac -cp "C:\Program Files\Java\jdk1.6.0_02\lib\ext\ssj.jar;." GenerateDemandArrivals.java java -cp "C:\Program Files\Java\jdk1.6.0_02\lib\ext\ssj.jar;." GenerateDemandArrivals
I already had ant installed on my Windows machine but so I said why not take advantage of it, I looked for simple ant file and found something quite close in apache ant manual.
After little bit more search and refinement here is what I came up with:
<project name="Sim" default="run" basedir=".">
<description>
simple example build file
</description>
<!-- set global properties for this build -->
<property name="src" location="."/>
<property name="build" location="."/>
<path id="project.class.path">
<pathelement path="C:\Program Files\Java\jdk1.6.0_02\lib\ext\ssj.jar"/>
<pathelement path="."/>
</path>
<target name="run"
description="run the program"
depends="compile">
<java classname="GenerateDemandArrivals">
<classpath refid="project.class.path"/>
</java>
</target>
<target name="compile"
description="compile the source " >
<!-- Compile the java code from ${src} into ${build} -->
<javac srcdir="${src}" destdir="${build}">
<classpath refid="project.class.path"/>
</javac>
</target>
</project>
Now I can simple type ant (while I am in the directory containing the build.xml file above) after any changes to the file and it compiles and run my code.
I feeling ashamed of feeling good and writing about such basic stuff but I am thinking there would be another person like me (could be myself at some future point where I have to leave my lovely Ruby world and come back to Java out of necessity).
JRuby: Stochastic Simulation with SSJ
I needed to work on a simulation project that involved working with the SSJ java library.
If I haven’t mentioned before I love Ruby (the programming language) and Java, though a tool in my toolkit, does not evoke the same feeling.
So my first reaction was I can use JRuby to benefit from the existing java library and beauty of Ruby.
The SSJ library comes with a comprehensive sets of examples and documentation. I started exploring that.
Fortunately at RubyConf 2008 with some help from Charles Nutter I was able to install jruby on my laptop, getting jruby installed and running was not trivial. I wish somebody can do a one click installer for jruby same as they did for Ruby and Rails.
I downloaded SSJ and unzipped the downloaded file containing jar files into C:\Program Files\Java\jdk1.6.0_02\lib\ext directory. Added the CLASSPATH variable in my windows environment set to this directory.
I tried the following example from examples.pdf
import umontreal.iro.lecuyer.rng.*;
import umontreal.iro.lecuyer.stat.*;
public class Collision {
int k; // Number of locations.
int m; // Number of items.
double lambda; // Theoretical expectation of C (asymptotic).
boolean[] used; // Locations already used.
public Collision (int k, int m) {
this.k = k;
this.m = m;
lambda = (double) m * m / (2.0 * k);
used = new boolean[k];
}
// Generates and returns the number of collisions.
public int generateC (RandomStream stream) {
int C = 0;
for (int i = 0; i < k; i++) used[i] = false;
for (int j = 0; j < m; j++) {
int loc = stream.nextInt (0, k-1);
if (used[loc]) C++;
else used[loc] = true;
}
return C;
}
// Performs n indep. runs using stream and collects statistics in statC.
public void simulateRuns (int n, RandomStream stream, Tally statC) {
statC.init();
for (int i=0; i<n; i++) statC.add (generateC (stream));
statC.setConfidenceIntervalStudent();
System.out.println (statC.report (0.95, 3));
System.out.println (" Theoretical mean: " + lambda);
}
public static void main (String[] args) {
Tally statC = new Tally ("Statistics on collisions");
Collision col = new Collision (10000, 500);
col.simulateRuns (100000, new MRG32k3a(), statC);
}
}
I compiled and ran the Java code above to ensure I was getting the same results using the following steps:
javac -cp "C:\Program Files\Java\jdk1.6.0_02\lib\ext\ssj.jar" Collision.java java -cp "C:\Program Files\Java\jdk1.6.0_02\lib\ext\ssj.jar;C:\code\ruby\jruby" Collision
As you can notice my code was in the C:\code\ruby\jruby directory and I was in this directory when I ran the above program.
Here is the same code example in jruby code:
require 'java'
require 'ssj.jar'
import 'umontreal.iro.lecuyer.rng.RandomStream'
import 'umontreal.iro.lecuyer.stat.Tally'
import 'umontreal.iro.lecuyer.rng.MRG32k3a'
class Collision
def initialize(k, m)
@k = k
@m = m
@lambda = m * m / (2.0 * k)
@used = Array.new(k, false)
end
def generate_c(stream)
c = 0
@k.times { |i| @used[i] = false }
@m.times do |j|
loc = stream.nextInt(0, @k-1)
if @used[loc]
c += 1
else
@used[loc] = true
end
end
return c
end
def simulate_runs(n, stream, stat_c)
stat_c.init
n.times { stat_c.add(generate_c(stream)) }
stat_c.setConfidenceIntervalStudent()
puts stat_c.report(0.95, 3)
puts " Theoretical mean: #{@lambda} "
end
def self.run
stat_c = Tally.new("Statistics on collision")
col = Collision.new(10000,500)
col.simulate_runs(100000, MRG32k3a.new, stat_c)
end
end
Collision.run
The jruby code runs comparatively slower (my feeling is 5 times slower but I will measure and add it later). I am using the latest available jruby 1.1.5 at the time of this post.
Update: The JRuby version takes around 13 minutes while the equivalent Java code above takes 17 seconds. This seems to be unacceptable performance, I am not sure if this is specific to SSJ or other libraries too. If I understand correctly, JRuby should be running inside the JVM and when I call SSJ library that is the same Java code that runs so I don’t understand why it is so slow unless I have used some technique that slows down my JRuby version.
MySQL: Restarting MySQL on Ubuntu
/etc/init.d/mysql start
OR
/etc/init.d/mysql restart
Source:
Mysql Database Server Installation and Configuration in Ubuntu
MySQL : Saving a ton of space on old tables
Here is a nice article about using MySQL (> 5) archive engine.
The MySQL 5.0 Archive Storage Engine
If you have some tables that has mostly read-only for data retention purposes then you can do something like this to drastically reduce the disk spaced used by that table:
alter table MY_OLD_TABLE drop primary key, engine=archive;
note the archive table does not have primary key and dropping the primary key in this way prevents from rebuilding the table twice (compared to the case where you issue two alter table, one to drop key and second to switch engine.
MySQL : Information about tables
desc information_schema.tables;
+-----------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------------+--------------+------+-----+---------+-------+ | TABLE_CATALOG | varchar(512) | YES | | NULL | | | TABLE_SCHEMA | varchar(64) | NO | | | | | TABLE_NAME | varchar(64) | NO | | | | | TABLE_TYPE | varchar(64) | NO | | | | | ENGINE | varchar(64) | YES | | NULL | | | VERSION | bigint(21) | YES | | NULL | | | ROW_FORMAT | varchar(10) | YES | | NULL | | | TABLE_ROWS | bigint(21) | YES | | NULL | | | AVG_ROW_LENGTH | bigint(21) | YES | | NULL | | | DATA_LENGTH | bigint(21) | YES | | NULL | | | MAX_DATA_LENGTH | bigint(21) | YES | | NULL | | | INDEX_LENGTH | bigint(21) | YES | | NULL | | | DATA_FREE | bigint(21) | YES | | NULL | | | AUTO_INCREMENT | bigint(21) | YES | | NULL | | | CREATE_TIME | datetime | YES | | NULL | | | UPDATE_TIME | datetime | YES | | NULL | | | CHECK_TIME | datetime | YES | | NULL | | | TABLE_COLLATION | varchar(64) | YES | | NULL | | | CHECKSUM | bigint(21) | YES | | NULL | | | CREATE_OPTIONS | varchar(255) | YES | | NULL | | | TABLE_COMMENT | varchar(80) | NO | | | | +-----------------+--------------+------+-----+---------+-------+
-- now you can do this or some such select table_name, engine from information_schema.tables where table_name like "MY_TABLE%";
