life.i.think -

What is Twitter to Me?
Scribbled on May 30th. 6 comments.

I’ve been thinking about this quite a bit lately, especially after several discussions with people around the topic at SxSW. It’s a subject of particular interest to be because I find it easy to verbalize but hard to convey digitally, especially within one’s attention span. As Amy Hoy recently said … “Brevity is the soul of twit.”

When I first came out to San Francisco to visit Jesse, a long time friend, I hadn’t planned on taking a job. In fact, my plan was to move from Lexington to South America. It would take a great bit to divert me from my chosen path, and to my surprise, diverted I was.

For a long time I’ve found great value in ‘the little things.’ Smiling at someone on the street, having a small conversation at the line at the market, dot, dot, dot. These are the things from which we build our character, our selves. We take the little things we see in each other that touch a particular emotion and absorb them into our own personas. I saw this potential within Twitter, except on an immensely greater scale. Of note, at the time I wasn’t ‘into’ social networks. The only one I even belonged to was Flickr. I had no MySpace or Facebook, but this little thing called Twitter somehow stuck me as different. A small chord that played and whispered: I do not replicate or improve on what has already been. I am revolution.

Revolution (n): a drastic and far-reaching change in ways of thinking and behaving.

I’ve now been working on Twitter for the better part of a year and a half. Interestingly, it’s become a posterchild for the discussion of scaling social internet architecture, yet what I have found equally as interesting is the social implications that have emerged. Something unexpected. Something new. Twitter changes the way I interact with people on some fundamental level! I have friends that both are, and are not on Twitter. The ways in which I communicate with them are actually beginning to differ. I should note that this is not negative, but there is a divergence. Twitter provides so much context into people’s lives. I know things that people are doing and thinking that I wouldn’t otherwise, either because they were too small to bring up in conversation, or weren’t appropriate to bring up in a later encounter based on time passed. Plus, we have way to many little things happening in our lives to discuss them all with others.

Enter Twitter. I find myself framing conversations based on microcontext { not to be confused with microbloging, which is a word I loathe }. These bits of personal information, these Twitters. Suddenly I find I have much more personal context around a conversation that might otherwise diverge to one of those “So, how are things?” interactions. This is a drastic and fundamental change in the way in which we communicate. All the little things! The smiles. The laughs. The tears. Walking the dog. Feeding your cat. Eating a banana. Having a baby. These are what Twitter are to me.

postscript.
It’s funny that within all the debate and negativity surrounding how to scale a platform, it’s the underlying ideals of Twitter, not the technical implications, that are truly important. These are the things that will survive and ultimately make our lives better. People get so swept up in the technical work, that they miss the little details that are so wonderfully important. Isn’t that the embodiment of what Twitter is all about?

Ruby GetText Memory Leaks
Scribbled on May 23rd. 3 comments.

Today I was working on some revisions to Twitter’s Protected User stack { or Project: PrivateParts as I like to refer to it in my head }, when I had to make some string changes to emails being sent to users. Offhandedly, I ran across this bug report on RubyForge.

HOLY HELL IN A HANDBASKET!

The Ruby GetText 1.90.0 gem uses an instance of a Class Object as a key to a hash! Steven Rusitschka noticed this and noted that “the memory Mongrel used increased by 1 MB with each request it served …”

To avoid any possibility of a reader absentmindedly skipping over that last quote from Steven, I present it for you again in vibrant color and with extra bold.

“the memory Mongrel used increased by 1 MB with each request it served …”

Okay. Deep breaths. Before everyone goes around MonkeyPatching GetText, there’s a new release out, 1.91.0. Let’s look and see if a fix has been put into place.

Old: 1.90.0


def bound_target(klass = self) # :nodoc:
  if cached?
    if @@__cache_bound_target[klass] # <---- Yowza!
      return @@__cache_bound_target[klass]
    end
  end

  ret = nil
  if klass.kind_of? Class or klass.kind_of? Module
    ret = klass
  else
    ret = klass.class
  end
  ret = GetText if ret.name =~ /^\#<|^$/
  @@__cache_bound_target[klass] = ret
  ret
end

New: 1.91.0


def bound_target(klass = self) # :nodoc:
  id = klass.object_id
  if cached?
   tgt = @@__cache_bound_target[id] # <---- Ahhh, much better.
   return tgt if tgt
  end

  ret = (klass.kind_of? Module) ? klass : klass.class
  if ret.name =~ /^\#<|^$/ or ret == GetText
     #GC for dead object_ids.
     ret = Object
     if @@__cache_bound_target.size > CACHE_BOUND_TARGET_MAX_SIZE
        @@__cache_bound_target.clear
     end
  end
  @@__cache_bound_target[id] = ret
  ret
end

The GetText gem is now using the object_id as the cache key. To say the least, this has made a huge difference in the memory profile of Twitter’s Mongrels. Not much more to say here. If you’re running GetText with Rails and are find yourself a bit leaky, check your gem version. You may be due for an upgrade!

GetText, Rails, Authentication, Japanese Firefox, Setting the Language Manually
Scribbled on May 14th. 2 comments.

There occurs an interesting problem when you are using Ruby’s GetText to translate a site that requires authentication.

Most likely, you have a before_filter that calls a method that authenticates the user for certain methods. Within this filter you probably have something like:


user = (attempt_oauth or
  attempt_basic_auth or
  attempt_session_auth or
  attempt_cookie_auth or
  (@authentication_attempt = nil))

If the user has clicked “Remember Me” and stored an cookie locally, than we’re going to validate the user using attempt_cookie_auth. But what if the user has a default language set (stored most likely in the database). We’ll assume there is a method user.lang that returns the default language or nil.

Your first attempt at setting this here may be this one.

NOTE: THIS IS WRONG!


# If the user has a default lang, set it here.
if u && u.lang && (LANGUAGE_CODES.include?(u.lang))
  cookies[:lang] = u.lang unless cookies[:lang]
end

That sets the cookie for the user, but the GetText stack has already been invoked, so the language will be that sent over by the browser. Refreshing the page will cause GetText to pick up the cookie value and render the proper language.

Next, we try using set_locale (GetText.set_locale) to set the language manually, which seems like a perfectly reasonable option.

NOTE: THIS IS WRONG!


# If the user has a default lang, set it here.
if u && u.lang && (LANGUAGE_CODES.include?(u.lang))
  cookies[:lang] = u.lang unless cookies[:lang]
  set_locale u.lang
end

Why is this bad? The set_locale method persists for the life of the Ruby instance (in this case, Mongrel), not the session. This means that there will be a literal battle for contention over which language to use.


User A sets the language to JA and the page renders JA.
User B sets the language to EN and the page renders JA.
User A sets the language to JA and the page renders EN.
You see the problem! We want a clean slate at the beginning of each request so GetText has no pre-conceived notions about what the language should be. Hrm … let’s try something.

NOTE: THIS IS WRONG!


# If the user has a default lang, set it here.
if u && u.lang && (LANGUAGE_CODES.include?(u.lang))
  cookies[:lang] = u.lang unless cookies[:lang]
  set_locale u.lang
else
  set_locale nil
end

Note … to be thorough, and because the authentication stack is not called for all methods, we also add this in our application.rb


  before_init_gettext :set_default_locale
  def set_default_locale; set_locale nil; end

Here, everything seems to work! We can even write a test to verify that the contention above doesn’t occur.


def test_lang_should_be_set_on_a_per_session_basis
  bob.lang = 'ja'
  assert bob.save
  bob.reload

  assert_equal 'ja', bob.lang
  post '/sessions', {:username_or_email => bob.screen_name, :password => 'foo'}
  assert_response :redirect
  follow_redirect!
  assert_response :redirect
  follow_redirect!
  assert_response :success
  assert_equal 'ja', Locale.current.language

  phoenix.reload
  assert_equal nil, phoenix.lang
  post '/sessions', {:username_or_email => phoenix.screen_name, :password => 'foo'}
  assert_response :redirect
  follow_redirect!
  assert_response :redirect
  follow_redirect!
  assert_response :success
  assert_equal 'en', Locale.current.language
end

But alas, download the Japanese version of Firefox and visit the page.

The browser is hungry for UTF-8 data. As you can see, the part of the page in which we set the locale manually using set_locale is being pushed as SHIFT-JIS. The blue highlighted area is actually outside of the application.rb controller stack, so is sent via GetText’s default assumptions: UTF-8.

But this isn’t a problem in Safari or Internet Explorer. Why? Let’s look at the value of HTTP_ACCEPT_LANGUAGE.


Firefox :   "HTTP_ACCEPT_CHARSET" => "EUC-KR,utf-8;q=0.7,*;q=0.7" 
Safari  :   "HTTP_ACCEPT_CHARSET" =>  ???

Safari actually doesn’t pass one, while Firefox gives precedence EUC-KR (effectively SHIFT-JIS) over UTF-8.

Sigh. Things are looking grim. Back to the drawing board. Let’s look at the order of precedence for how the Rails GetText integration determines the langauge.


The language passed to GetText.bindtextdomain.
The lang query param. ( url?lang=foo )
The lang cookie.
The value of HTTP_ACCEPT_LANGUAGE passed by the browser.
The default (English).

Aha! GetText.bindtextdomain! Looking at the RDocs and source, this is not only called within the init_gettext method, but can be set on a per session basis without mucking with default language settings!

Let’s try!

NOTE: THIS IS CORRECT! CELEBRATE!


# If the user has a default lang, set it here.
if u && u.lang && (LANGUAGE_CODES.include?(u.lang))
  cookies[:lang] = u.lang unless cookies[:lang]
  GetText.bindtextdomain("Twitter", :locale => u.lang)
end

So finally, we can set the language on a request manually after the GetText stack has already been invoked. Whew!