[Ruby] Parallel migration

[Alex Vollmer]

Well the only reason I even brought this up was that we have multi-proc
machine that was  maxed on one proc and the other procs were totally idle.
The migration in question was changing the column type for several tables
which was done with a series of ALTER TABLE commands.

I can't imagine that we would run this migration more than a few times over
such a large dataset (we have several environments), but it had run for
nearly forty-five minutes before we decided to kill it and do it by hand.
When you're trying to get a release out, it doesn't look too cool to have a
migration that takes longer than your lunch-hour. ;-)

In retrospect a faster way to do this would be to create new tables with the
correct column types and select rows from the old tables into them. Then we
would drop the old tables and rename the new ones. Since this is a much
faster operation in Postgres than a vanilla ALTER TABLE, the question of
doing these in parallel would be largely irrelevant.

I brought this up mostly because I have a DBA who is very suspicious of the
entire migration model and wants to control all schema definition in ERWin
(note, nobody else is running Windows). While I won't blindly support
ActiveMigrations just because it's Ruby, I want to push it as far as I can
since it's the best solution I've seen so far to rapid evolution of a
database schema. Personally, I think reaching for the
parallel-threads-hammer is a bit premature, but I was curious what others
experiences were.

Thanks,

Alex V.



On 1/19/07, BenJamin Prater <bprater at gmail.com> wrote:
>
> I can't imagine needing to alter tables on a continual basis. Wouldn't
> this be a one-shot deal where you could live with the time necessary
> to alter the table?
>
> Even if you could parallel alter (you could always do this on the
> command line and bypass migrations), it may not go any quicker since
> your cpu is going to be fairly saturated by the previous request.
>
> Ben
> _______________________________________________
> Ruby at zenspider.com - Seattle.rb non-commercial list
> http://www.zenspider.com/seattle.rb
> http://www.zenspider.com/mailman/listinfo/ruby
>



-- 
"I think there are only three things America will be known for 2,000 years
from now when they study this civilization: the Constitution, jazz music,
and baseball."
--Gerald Early