Initial import from MonetDB-0bcd960808f4 checkout

Changelog

+* Tue 30 Jun 2009 22:14:38 CEST
+Added test cases for the standalone driver (ruby unit test).
+
+* Thu 25 Jun 2009 17:31:02 CEST
+Fixed a major bug that resulted in data corruption;
+MAPI implementation code cleanup and bug fixes;
+Support to ruby 1.9;
+Both the standalone driver and activerecord now support transactions; nested transactions
+are simulated via savepoints in activerecord.
+Added a Rakefile and script to run activerecord unit test suite.
+Type conversion in the standalone driver is now performed after data has been retrieved and can be executed on single fields.
+
+* Mon 25 May 2009 17:52:01 CEST
+Imported last week changes (protocol 9 support, parametrized the connection options in the activerecord adapter, fixed a bug in the auth protocol v8).
+Fixed a bug in the mapi protocol that resulted in data loss (wrong handling of TRANSACTIONS).
+Began to port the activerecord test suite to monetdb (not all test cases can be performed).
+Removed an unneeded file ('lib/MonetDBStatement.rb') from the gemspec of the standalone driver (the feature will be moved to HEAD).
+Began to port the driver to ruby 1.9.
+Removed *.gem files from cvs.
+
+* Mon 18 May 2009 15:22:31 CEST
+Fixed bugs that prevented the correct working of activerecords' migration;
+The activerecord connector now supports insertion, update and alter table operations;
+Type casting is working in activerecord;
+Added a rubygem and rakefile for activerecord-monetdb-adapter;
+Added a new usage example for activerecord to the README file;
+Added an example directory to the cvs tree;
+The driver now correctly works with merovingian.
+
+
+* Sat  9 May 2009 15:58:36 CEST
+Fixed bugs with the query processing in the standalone driver;
+Added INSERT and UPDATE methods in the activerecord connector.
+
+* Thu  7 May 2009 17:03:01 CEST
+
+Added a check against the protocol version during authentication;
+Imported the activerecord code (available under adapter/).

Makefile.ag

+# The contents of this file are subject to the MonetDB Public License
+# Version 1.1 (the "License"); you may not use this file except in
+# compliance with the License. You may obtain a copy of the License at
+# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
+#
+# Software distributed under the License is distributed on an "AS IS"
+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
+# License for the specific language governing rights and limitations
+# under the License.
+#
+# The Original Code is the MonetDB Database System.
+#
+# The Initial Developer of the Original Code is CWI.
+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
+# Copyright August 2008-2011 MonetDB B.V.
+# All Rights Reserved.
+
+SUBDIRS = adapter
+
+gem_monetdb = {
+	FILES = ruby-monetdb-sql-0.1.gemspec
+	DIR = $(prefix)/$(RUBY_DIR)
+}
+
+EXTRA_DIST = Changelog README TODO ruby-monetdb-sql-0.1.gemspec
+EXTRA_DIST_DIR = examples lib

README

+== Standalone driver ==
+This directory contains the a ruby interface to monetdb5 
+written in pure ruby.
+
+lib/MonetDB.rb
+lib/MonetDBConnection.rb
+lib/MonetDBStatements.rb
+lib/MonetDBData.rb
+lib/MonetDBExceptions.rb
+lib/hasher.rb
+lib/demo.rb: demo application how to interact with the database
+
+ruby-monetdb-sql-0.1.gemspec: make file for rubygems
+
+doc/: rubydoc in HTML format
+
+== Installation ==
+
+The standalone monetdb driver can be installed using the RubyGems Package Manager.
+
+First build a gem file starting from the gemspec configuration:
+
+$ gem build ruby-monetdb-sql-0.1.gemspec
+
+Then install with the command:
+
+$ gem install ruby-monetdb-sql-0.1.gem
+
+== Usage ==
+To use the standalone driver import the 'MonetDB' class and 'rubygems' (in case you installed it using gems).
+
+A typical sequence of events is as follows:
+Invoke query using the database handle to send the statement to the server and get back a result set object.
+
+A result set object  has methods for fetching rows, moving around in the result set, obtaining column metadata, and releasing the result set.
+Use a row fetching method such as fetch_row or an iterator such as each to access the rows of the result set.
+If you want a count of the number of rows in the result set: invoke 'num_rows' method.
+Invoke 'free' to release the result set. 
+
+== Example ==
+
+require 'MonetDB'
+
+db = MonetDB.new
+db.connect(user = "monetdb", passwd = "monetdb", lang = "sql", host="127.0.0.1", port = 50000, db_name = "demo", auth_type = "SHA1")
+
+# set type_cast=true to enable MonetDB to Ruby type mapping
+res = db.query("SELECT * from tables;", type_cast = false)
+
+#puts res.debug_columns_type
+
+puts "Number of rows returned: " + res.num_rows.to_s
+puts "Number of fields: " + res.num_fields.to_s
+
+
+# Get the columns' name
+col_names = res.name_fields
+
+
+# Iterate over the record set and retrieve on row at a time
+puts res.fetch
+while row = res.fetch do
+  printf "%s \n", row
+end
+
+# Release the result set.
+res.free
+
+# Disconnect from server
+db.close
+
+See lib/demo.rb and the MonetDBDatar class documentation for more examples.
+
+
+
+== ActiveRecord connector adapter ==
+Active Record connects business objects and database tables to create a persistable domain model where logic and data are presented in one wrapping. It‘s an implementation of the object-relational mapping (ORM) pattern.
+
+Required files:
+
+adapter/lib/active_record/monetdb_adapter.rb
+
+Usage example follows:
+require 'active_record'
+
+ActiveRecord::Base.logger = Logger.new(STDERR)
+ActiveRecord::Base.colorize_logging = true
+
+ActiveRecord::Base.establish_connection(
+        :adapter => "monetdb",
+        :host => "localhost",
+        :database => "demo"
+)
+
+# Create a new table
+class AddTests < ActiveRecord::Migration
+  def self.up
+    create_table :tests do |table|
+        table.column :name, :string
+        table.column :surname, :string
+    end
+  end
+
+  def self.down
+   drop_table :tests
+  end
+
+end
+
+AddTests.up
+
+# Migration: add a column name with a default value
+class AddAge < ActiveRecord::Migration 
+  def self.up
+    add_column :tests, :age, :smallint, :default => 18
+  end
+
+  def self.down
+    remove_column :tests, :age
+  end
+
+end
+
+class Test < ActiveRecord::Base
+end
+
+# Insert an entry in the table
+Test.create(:name => 'X', :surname => 'Y')
+
+# add a column
+AddAge.up
+
+# return the first result of the query SELECT * from tables
+row = Test.find(:first)
+printf "SELECT * from tests LIMIT 1:\n"
+printf "Name: %s, Surname: %s, Age: %s\n", row.name, row.surname, row.age
+
+# Drop the table
+AddTests.down
+
+== Rubygem ==
+
+The standalone ruby driver can be distributed as a ruby gem.
+A gem file is already available; however, it can be generated 
+starting from the ruby-monetdb-sql-0.1.gemspec file:
+
+$ gem build ruby-monetdb-sql-0.1.gemspec
+
+To install the file run the command:
+
+$ gem install ruby-monetdb-sql-0.1.gem
+
+Documentation in ri and html format will be generated and installed as well
+

TODO

+* test and improve utf8 and type conversion
+
+* documentation cleanup
+
+* OSM on rails demo (slowed down due to third party plugins requirements)

adapter/Makefile.ag

+# The contents of this file are subject to the MonetDB Public License
+# Version 1.1 (the "License"); you may not use this file except in
+# compliance with the License. You may obtain a copy of the License at
+# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
+#
+# Software distributed under the License is distributed on an "AS IS"
+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
+# License for the specific language governing rights and limitations
+# under the License.
+#
+# The Original Code is the MonetDB Database System.
+#
+# The Initial Developer of the Original Code is CWI.
+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
+# Copyright August 2008-2011 MonetDB B.V.
+# All Rights Reserved.
+
+gem_adapter = {
+	FILES = activerecord-monetdb-adapter-0.1.gemspec
+	DIR = $(prefix)/$(RUBY_DIR)
+}
+
+EXTRA_DIST = activerecord-monetdb-adapter-0.1.gemspec
+EXTRA_DIST_DIR = active_record lib

adapter/activerecord-monetdb-adapter-0.1.gemspec

+Gem::Specification.new do |s|
+   s.required_ruby_version = '>= 1.8.0'
+   s.name = %q{activerecord-monetdb-adapter}
+   s.version = "0.1"
+   s.date = %q{2009-05-18}
+   s.authors = ["G Modena"]
+   s.email = %q{gm@cwi.nl}
+   s.summary = %q{ActiveRecord Connector for MonetDB}
+   s.homepage = %q{http://monetdb.cwi.nl/}
+   s.description = %q{ActiveRecord Connector for MonetDB built on top of the pure Ruby database driver}
+   s.files = [ "lib/active_record/connection_adapters/monetdb_adapter.rb" ]
+   s.has_rdoc = true
+   s.require_path = 'lib'
+   s.add_dependency(%q<activerecord>, [">= 2.3.2"])
+   s.add_dependency(%q<ruby-monetdb-sql>, [">= 0.1"])
+   # placeholder project to avoid warning about not having a rubyforge_project
+   s.rubyforge_project = "nowarning"
+end

doc/rails.txt

+= MonetDB on rails =
+
+This file documents the steps necessary to make rails working with MonetDB.
+Database management in rails is performend via the ActiveRecord (ORM) adapter (http://api.rubyonrails.org/files/vendor/rails/activerecord/README.html).
+
+The ruby/monetdb driver comes packaged as:
+
+1) a standalone driver (only depends on ruby 1.8 and ruby 1.9);
+2) and activerecord connector built on top of the standalone driver.
+
+The following steps assumes that you already have obtained and installed a working copy of the rails framework and the 'rubygems' package manager.
+
+For more informations about rubygems please refer to: http://rubyforge.org/projects/rubygems/
+
+== Installation == 
+
+On rpm base systems both the activerecord and standalone driver should be installed via the monetdb-clients package.
+Please note that the building and installing process do not require root privileges.
+
+
+=== Install the standalone driver ===
+
+If you are using a source code version of monetdb cd into the driver directory located at './clients/src/ruby'
+
+First build a gem file starting from the given gemspec:
+
+$ gem build ruby-monetdb-sql-0.1.gemspec
+
+and install the resulting gem with the command:
+
+$ gem install --local ruby-monetdb-sql-0.1.gem
+
+
+=== Install the activerecord connector ===
+
+If you are using a source code version of monetdb cd into the activerecord adapter directory located at './clients/src/ruby/adapter'
+
+First build a gem file starting from the given gemspec:
+
+$ gem build activerecord-monetdb-adapter-0.1.gemspec
+
+and install the resulting gem with the command:
+
+$ gem install --local activerecord-monetdb-adapter-0.1.gem
+
+
+== Configuration ==
+
+Create a new rails application with the 'rails' command and cd into it:
+
+$ rails myapp
+$ cd myapp
+
+This will create the usual Rails folder structure.
+
+In order to start using rails with MonetDB as a backend we need to properly configure the databases by editing the file 'config/database.yml'
+
+This is a sample configuration used to connect to a database named 'test':
+
+  adapter: monetdb
+  username: monetdb 
+  password: monetdb
+  hostname: localhost
+  port:    50000
+  database: test
+
+
+Where adapter is the name of the activerecord connector you want to use (in this case 'monetdb').
+
+The driver works either by directly connecting to a mserver5 instance or through merovingian. Make sure to create the database ('test') before proceeding.
+
+Once the database.yml file as been properly configured proceed by creating the Controller files either manually or by using the scaffolding script:
+
+$ ./script/generate scaffold
+
+Operations on the database are performed in the fashion of rake tasks; once the Controllers have been properly created and customized, schemas can be generated with the  task 'migrate':
+
+$ rake db:migrate
+
+The current schema version number can be obtained with the task 'version':
+
+$ rake db:version
+
+It is possible to revert to previous schemas definitions via the 'rollback' task:
+
+$ rake db:rollback
+
+In the same way it is possible to drop all generated schemas with the 'drop' task:
+
+$ rake db:drop:all
+
+== References ==
+
+What follows is a list of tutorials about getting started with rails.
+
+1) http://www.akitaonrails.com/2007/12/12/rolling-with-rails-2-0-the-first-full-tutorial
+2) http://oreilly.com/ruby/archive/rails.html
+3) http://developer.apple.com/tools/rubyonrails.html

examples/activerecord.rb

+# The contents of this file are subject to the MonetDB Public License
+# Version 1.1 (the "License"); you may not use this file except in
+# compliance with the License. You may obtain a copy of the License at
+# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
+#
+# Software distributed under the License is distributed on an "AS IS"
+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
+# License for the specific language governing rights and limitations
+# under the License.
+#
+# The Original Code is the MonetDB Database System.
+#
+# The Initial Developer of the Original Code is CWI.
+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
+# Copyright August 2008-2011 MonetDB B.V.
+# All Rights Reserved.
+
+require 'rubygems'
+require 'active_record'
+
+ActiveRecord::Base.logger = Logger.new(STDERR)
+ActiveRecord::Base.colorize_logging = true
+
+ActiveRecord::Base.establish_connection(
+        :adapter  => "monetdb",
+        :host     => "localhost",
+        :database => "test",
+        :username => "monetdb",
+        :password => "monetdb",
+        :hostname => "localhost",
+        :port     => 50000
+)
+
+# Create a new table
+class AddTests < ActiveRecord::Migration
+  def self.up
+    create_table :tests do |table|
+        table.column :name, :string
+        table.column :surname, :string
+    end
+  end
+
+  def self.down
+   drop_table :tests
+  end
+
+end
+
+AddTests.up
+
+# Migration: add a column name with a default value
+class AddAge < ActiveRecord::Migration 
+  def self.up
+    add_column :tests, :age, :smallint, :default => 18
+  end
+
+  def self.down
+    remove_column :tests, :age
+  end
+
+end
+
+class Test < ActiveRecord::Base
+end
+
+# Insert an entry in the table
+Test.create(:name => 'X', :surname => 'Y')
+
+# add a column
+AddAge.up
+
+# return the first result of the query SELECT * from tables
+row = Test.find(:first)
+printf "SELECT * from tests LIMIT 1:\n"
+printf "Name: %s, Surname: %s, Age: %s\n", row.name, row.surname, row.age
+
+# revert the added column
+AddAge.down
+
+# Drop the table
+AddTests.down

examples/standalone.rb

+# The contents of this file are subject to the MonetDB Public License
+# Version 1.1 (the "License"); you may not use this file except in
+# compliance with the License. You may obtain a copy of the License at
+# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
+#
+# Software distributed under the License is distributed on an "AS IS"
+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
+# License for the specific language governing rights and limitations
+# under the License.
+#
+# The Original Code is the MonetDB Database System.
+#
+# The Initial Developer of the Original Code is CWI.
+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
+# Copyright August 2008-2011 MonetDB B.V.
+# All Rights Reserved.
+
+require 'rubygems'
+require 'MonetDB'
+
+db = MonetDB.new
+
+db.connect(user = "monetdb", passwd = "monetdb", lang = "sql", host="127.0.0.1", port = 50000, db_name = "test", auth_type = "SHA1")
+
+# set type_cast=true to enable MonetDB to Ruby type mapping
+res = db.query("select * from tables");
+
+puts "Number of rows returned: " + res.num_rows.to_s
+puts "Number of fields: " + res.num_fields.to_s
+
+
+# Get the columns' name
+#col_names = res.name_fields
+
+# Get the columns' type
+#col_types = res.type_fields
+
+###### Fetch all rows and store them
+#puts res.fetch_all
+
+
+# Iterate over the record set and retrieve on row at a time
+while row = res.fetch do
+  printf "%s \n", row
+end
+
+
+###### Get all records and hash them by column name
+#row = res.fetch_all_hash()
+
+#puts col_names[0] + "\t\t" + col_names[1]
+#0.upto(res.num_rows) { |i|
+#  puts row['id'][i] + "\t\t" + row['name'][i] 
+#}
+
+
+###### Iterator over columns (on cell at a time), convert the "id" field to a ruby integer value.  
+
+#while row = res.fetch_hash do
+#  printf "%s, %i\n", row["name"], row["id"].getInt
+#end
+
+###### Transactions
+
+db.query("START TRANSACTION")
+db.auto_commit(false)
+# create a savepoint
+db.save
+db.query("SAVEPOINT #{db.transactions} ;")
+
+# Modify the database
+db.query('CREATE TABLE test (col1 INT, col2 INT)')
+
+# Rollback to previous savepoint, discard changes
+
+db.query("ROLLBACK TO SAVEPOINT #{db.transactions}")
+# Release the save point
+db.release
+
+# Switch to auto commit mode
+db.auto_commit(true)
+
+# Deallocate memory used for storing the record set
+res.free
+
+db.close

lib/MonetDBExceptions.rb

+# The contents of this file are subject to the MonetDB Public License
+# Version 1.1 (the "License"); you may not use this file except in
+# compliance with the License. You may obtain a copy of the License at
+# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
+#
+# Software distributed under the License is distributed on an "AS IS"
+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
+# License for the specific language governing rights and limitations
+# under the License.
+#
+# The Original Code is the MonetDB Database System.
+#
+# The Initial Developer of the Original Code is CWI.
+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
+# Copyright August 2008-2011 MonetDB B.V.
+# All Rights Reserved.
+
+# Exception classes for the ruby-monetdb driver
+
+class MonetDBQueryError < StandardError
+  def initialize(e)
+    $stderr.puts e
+  end
+end
+
+class MonetDBDataError < StandardError
+  def initialize(e)
+    $stderr.puts e
+  end
+end
+
+class MonetDBCommandError < StandardError
+  def initialize(e)
+    $stderr.puts e
+  end
+end
+
+class MonetDBConnectionError < StandardError
+  def initialize(e)
+    $stderr.puts e
+  end
+end
+
+
+class MonetDBSocketError < StandardError
+  def initialize(e)
+    $stderr.puts e
+  end
+end
+
+class MonetDBProtocolError < StandardError
+   def initialize(e)
+     $stderr.puts e
+   end  
+end
\ No newline at end of file

lib/demo.rb

+# The contents of this file are subject to the MonetDB Public License
+# Version 1.1 (the "License"); you may not use this file except in
+# compliance with the License. You may obtain a copy of the License at
+# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
+#
+# Software distributed under the License is distributed on an "AS IS"
+# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
+# License for the specific language governing rights and limitations
+# under the License.
+#
+# The Original Code is the MonetDB Database System.
+#
+# The Initial Developer of the Original Code is CWI.
+# Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
+# Copyright August 2008-2011 MonetDB B.V.
+# All Rights Reserved.
+
+require 'MonetDB'
+
+db = MonetDB.new
+
+db.conn({ :user => "monetdb", :passwd => "monetdb", :port => 50000, :language => "sql", :host => "localhost", :database => "ruby_test", :auth_type => "SHA1" })
+
+# set type_cast=true to enable MonetDB to Ruby type mapping
+#res = db.query("select * from tables, tables, tables;")
+
+#db.query("DROP TABLE tests2 ")
+#db.query(" CREATE TABLE tests2 ( col1  varchar(255), col2 varchar(255)) " )
+
+#puts "Number of rows returned: " + res.num_rows.to_s
+#puts "Number of fields: " + res.num_fields.to_s
+
+# Get the columns' name
+# print res.name_fields
+
+###### Fetch all rows and store them
+
+#puts res.fetch_all
+
+
+# Iterate over the record set and retrieve on row at a time
+#puts res.fetch
+#while row = res.fetch do
+#  printf "%s \n", row
+#end
+
+
+###### Get all records and hash them by column name
+#row = res.fetch_all_hash()
+
+#puts col_names[0] + "\t\t" + col_names[1]
+#0.upto(res.num_rows) { |i|
+#  puts row['id'][i]
+#}
+
+
+###### Iterator over columns (on cell at a time)
+
+#while row = res.fetch_hash do
+#  printf "%s\n",  row["id"]
+#end
+
+# SQL TRANSACTIONS and SAVE POINTS
+
+
+
+db.query('DROP TABLE tests2')
+db.auto_commit(false)