Thursday, January 31, 2008

Primenet is 10 years old

It was 10 years ago today that I installed the new prime95 program that would connect to primenet to retrieve mersenne prime numbers for testing. I had been participating for over 2 years before primenet was created. Before primenet was created you sent an e-mail to George Woltman. George kept track of all of the prime numbers and would send you 3-4 primes that you would run. You would then e-mail the results in your results file from him to process back into his database where he tracked all of the results.

Prime net replaced all of this manual effort and allowed George to focus on maintaining and improving his FFT code. Prime95 is a program that takes advantage of CPU’s architecture, and pushes it to it’s limits. It is also one of the friendliest programs to install on your computer. This program plays nice with all of programs I have ever run in my 10+ years running it. I’ve never seen it interfere with other programs. It has also detected many CPU and RAM failures before they became failures that ended in catastrophe. This is a testament to George’s concise meticulous error trapping programming ways and truly only wanting to use free CPU cycles and nothing else. I’ve run other distributed programs and several of them tend to want to hog the CPU and not let go immediately.

Here are some helpful links to learn about Mersenne primes, and how you can contribute to discovering them.

http://www.mersenne.org/

http://www.mersenne.org/primenet/status.shtml

http://mersennewiki.org/index.php/Main_Page

Some quick stats for marking the 10 years:

As of today I’ve tested 293 Exponents

Totaling 656 Pentium 90 CPU years to the project + a couple of years prior to primenet.

Over the past 10 years it has taken me an average of 3-6 weeks to complete a test of a single prime depending upon the age of my computer.

A prime in the 39-40 million range takes about 10-11 Pentium 90 years. So in the space of 3-6 weeks today, I can do the work that it would take my P90 10-11 years to do. Amazing

Friday, January 11, 2008

First Ruby Script

I wrote my first ruby script today. I’ve been playing with rails and trying things out by example for the past few weeks., but this is the first time a wrote something that I actually needed to get something done at work. One of the projects I worked on last year was upgrading our imaging solution for AP. The imaging solution was over 3 major releases old. We were behind a total of 17 dot releases, which would get us to a point where we could jump to the current version. This old version that we were on had started to develop issues that just couldn’t be solved. One of the major issues was that some users could not view the images. As best as we could determine at the time, incompatibilities had arisen due to changes in Java. The core product was written for Java 1.2 or 1.3. Very old and very unsupported. The imaging solution and almost everything that it used from 3rd parties was no longer supported, but that is a different story.

Turns out it wasn’t the java it was the actual image file. Somewhere in the chain of scanning, and transmitting the image, some of the tif files have become slightly corrupted. I guess new versions of Java now check the file instead of just blindly displaying the image or displaying what it can display without so much of a warning that something is wrong..

So now I know that some of the image files are corrupt and need to be fixed. How do I get these corrupt files? The images are on an ftp server, and I have a query that gives me the file information based upon information stored on an invoice. Running a query and manually ftp’ing and retrieving the images is very time consuming. A single invoice could have several pages associated with it: Meaning several image files for a document since each page is stored as a separate image.

This would be a great place to try out ruby, perl is a pain to get working on a windows PC. Ruby turned out to be very easy to install. I did start off with two separate ruby scripts and then combined the two into a single script.

The first step was to try to ftp to the server and retrieve one of the files. Googling gave me this link from Ruby Docs . This helped me get to a point to ftp the file back. This saved some time, but it would be really helpful if I could query the Oracle Database and retrieve the files based on the query.

A query of “ruby oracle” brought back some very good articles. These articles are on Oracle’s site: Accessorize Oracle Database with Ruby , Ruby on Rails on Oracle: A Simple Tutorial , and Tips for Optimizing Rails on Oracle . I haven’t read all of the articles in their entirety, but it did narrow down what I had to do very quickly.

I downloaded Ruby/OCI8 at http://rubyforge.org/projects/ruby-oci8/ followed the directions for installing on a windows machine and was ready to try running a query. I took one of the example queries in one of the articles and modified it to meet my needs.

Connecting to the database wasn’t clear, and having text values contained within single quotes also gave me a few issues.

I’ve worked with perl’s DBI with other project’s so ruby’s was not that hard to figure out. The only issue I had was that some of the calls were not clear that they were calls. I’ve modified the connect line to reflect something more readable.

dbh = DBI.connect('DBI:OCI8:[DBINSTANCE]', 'user', 'password')

Ruby’s wiki had a How to Connect to Oracle page that was very helpful. My experience in the past with DB connection examples are that you always use the default DB name and have only one DB.

Once I got past the connect issue I ran into a new one where ruby wasn’t liking the single quote used in the query. This was easily resolved by double quoting the entire query.

Below is the end result. Start to finish including downloading and installing everything needed. 2 hours. This will save me hours of time now. With audit season coming on strong, the request of images that can’t be retrieved through the image solution are only going to increase.

--Start Script

require 'dbi'
require 'net/ftp'

dbh = DBI.connect('DBI:OCI8:[DBINSTANCE]', 'user', 'password')

rs = dbh.prepare("select FILENAME, substr(COMPLETE_FILENAME, 32,21) from [table]
where document_id = 952154")

rs.execute

#Connect to FTP Server to retrieve image files

ftp = Net::FTP.new('[ftp_server]')
ftp.debug_mode=true
ftp.login(user = "user", passwd = "password")

while rsRow = rs.fetch do #loop through query results

#retrieve image files returned by query
ftp.chdir(rsRow[1])
ftp.getbinaryfile(rsRow[0], File.basename(rsRow[0]))

end

rs.finish

dbh.disconnect

ftp.close

--End Script

Labels: ,