[Ruby] String concatenation performance in Rails partials

[Tom Lianza]

On 11/29/06, Eric Hodel <drbrain at segment7.net> wrote:
>
> On Nov 29, 2006, at 0113 , Tom Lianza wrote:
>
> > Evan and a few others saw me working on this problem tonight, and I
> > just
> > stumbled upon a development that was fairly surprising to me.
> >
> > Basically, I have a chunk of HTML I want to render, say, 50 times
> > on a page
> > (this is a 'search results' page).  Each of the objects looks very
> > similar,
> > so I use a partial, and call it 50 times. Rails has a convenient
> > way of
> > doing this:
> >
> > <%=render(:partial=>'list_item', :collection=>bookmark_list)%>
> > (bookmark_list is an array of 50 things)--------------^
> >
> > The problem is that this seemed to be taking a long time to
> > render.  The DB
> > query was quick, but the rendering took 2.5 seconds per request
> > (benchmark
> > of 6 consecutive requests):
> >     user     system      total        real
> >   2.270000   0.040000   2.310000 (  2.483692)
> >   2.230000   0.030000   2.260000 (  2.488522)
> >   2.230000   0.020000   2.250000 (  2.477388)
> >   2.250000   0.040000   2.290000 (  2.622520)
> >   2.250000   0.040000   2.290000 (  2.768218)
> >   2.270000   0.030000   2.300000 (  2.727301)
> >
> > Here's the thing that's really weird - I ripped out the code I was
> > looping
> > on (the ruby and html in the list_item partial) and replaced it
> > with totally
> > static code - no ruby at all.  The performance was virtually
> > *identical.*
> >
> > So, on a hunch, I took that partial (about 40 lines html) and
> > removed all of
> > the whitespace, so it was all on one line.  The performance changed
> > fairly
> > dramatically:
> >
> >      user     system      total        real
> >   1.500000   0.020000   1.520000 (  1.808402)
> >   1.470000   0.030000   1.500000 (  1.698254)
> >   1.470000   0.020000   1.490000 (  1.905468)
> >   1.460000   0.020000   1.480000 (  1.622462)
> >   1.500000   0.030000   1.530000 (  1.923870)
> >   1.490000   0.030000   1.520000 (  2.136822)
> >
> > Rendering time was cut by about 1/3.
> >
> > If I cut down the html so I'm rendering a tiny amount (like one
> > short line)
> > the whole thing screams.  So... this really smells to me like some
> > kind of
> > string concatenation problem/slowdown.  Is anyone familliar with
> > these kinds
> > of issues in Ruby?  Or, might there be an issue in the Rails code?
> > I had
> > never heard any recommendations that stripping whitespace out of
> > templates
> > improves performance in any way, so I feel like this kind of
> > behavior is not
> > by design.
>
> str = ""
> str << a
> str << b
> str << c
>
> is slower than
>
> str = []
> str << a
> str << b
> str << c
> str = str.join
>
> Because (among other reasons) String#<< works the garbage collector
> more than Array#join.  ERb/eruby uses the former when it should use
> the latter.
>

So, based on that information one of the things I tried was creating my own
string in a loop of 50 items, so that erb would only have to do the
concatentation once.  Surprisingly, that didn't change the performance
characteristics measurably.  What does change them dramatically is if I
build up the string of 50 things, and don't append that big chunk to the
page.

This code ran in under a half a second:

output = []
bookmark_list.each{|bm| output << render_to_string(:partial=>'list_item',
:object=>bm)}

And if I add this line at the end, the time jumps up to 3 seconds:
<%=output%>

So... it sems like that one, big string concatenation is what's sucking up
all of the time.  That makes me wonder if the issue doesn't have anything to
do with lots of concatenations, but it has to do with the fact that there
are two big strings being concatenated.  I did some graphing, and the time
seems to be linear with the amount of data in the output array.

Any ideas on what might be going on?  What seems to be taking all of the
time is literally that one line ( <%=output%> ).  It seems weird to me that
this is slow, since I imagine it's common for ActionView to have to assemble
multiple large chunks of HTML as it puts partials together.

Thanks!
Tom