Emulating Ruby’s “method_missing” in Python

I don’t pretend to be a huge fan of Ruby. That said, I can respect when a language has a feature that’s pretty damn neat and useful. For the uninformed, method_missing in Ruby is something like the following:

Obviously, this is a trick that should be used with caution. It can make for some unmaintainable code, as a class with many methods could get difficult to trace through and figure out just what the hell is happening. It can be put to good use, though – take an API wrapper, for instance. What’s it consist of? Generally, nothing more than the same function calls made over and over to various service endpoints.

Cool, let’s use this in Python!

I recently rewrote Twython to support OAuth authentication with Twitter (as of Twython 1.3). It ships with an example Django application to get people up and running quickly, and the adoption has been pretty awesome so far.

The funny thing about the Twython 1.3.0 release is that it was largely a rewrite of the entire library. It had become somewhat unwieldy, some odd two thousand lines of code with each API endpoint getting its own method definition. The only differing aspect of these calls is the endpoint URL itself. This is a perfect case for a method_missing setup – let’s catch the calls to non-existent methods, and grab them out of a dictionary mapping to every endpoint.

The source above is fairly well commented, but feel free to ask in the comments if you need further explanation. This resulted in a much more maintainable version of Twython – for each function that’s listed in a hash table, we can now just take any named parameter and url-encode/combine it. This makes Twython pretty API-change agnostic of the entire Twitter API. Pretty awesome sauce, no?

Tags: , ,

11 Responses to “Emulating Ruby’s “method_missing” in Python”

  1. […] This post was mentioned on Twitter by elg0nz and Ryan McGrath, Python UK. Python UK said: Emulating Ruby's “method_missing” in Python « Veno Designs http://bit.ly/duo8nV […]

  2. […] Emulating Ruby's “method_missing” in Python « Veno Designs […]

  3. […] an API library.I've done a full writeup with code samples here, if anyone's interested:http://venodesigns.net/2010/11/0…Insert a dynamic date here BIU     @     Edit […]

  4. mvrak says:

    If you really like it then submit a PEP. If the community agrees then you’ve just strengthened Python. There are more coders available than idea people!

    Also, why call it “emulating”? It is really “implementing” unless you are making an incomplete copy.

  5. Ryan says:

    Hey there!

    Sorry, I just got time to circle back to this… I actually responded to your comment over on r/python, but I’ll put it here as well for the sake of complete-ness. Thanks for commenting!

    I personally don’t like it being a freely-available trick, as I think it introduces rather sloppy coding practices in general. There’s very few cases where this even makes sense, at least in my opinion, hence why I’d probably never submit a PEP on it. Cheers, though.

  6. dgx says:

    Well, it’s not like method_missing you know.
    Because method_missing is called only if Ruby doesn’t find the method required.
    In your implementation, __getattr__ is called all the time.
    It means that if I want to do
    print twitter
    it will fail, because there is no method __str__ nor __repr__ inside method_dictionary.

    An ugly way to pass through this would be to copy every standard method inside it, but well… it’s pretty damn ugly isn’t it?

  7. Ryan says:

    You have a good (and valid) point; at a glance I still think of it as being the same general idea, but the execution path is different, yes.

    I bet there’s a way to do it more like method_missing beyond copying every standard method. If I ever get free time, I’d look into it for the hell of it.

  8. This is a very cool trick, and I just implemented it to add some method missing magic to a project. I cannot seem to find an adequate explanation for _why_ it works though, specifically the __get__ part. Everything I see documented about __get__ talks about descriptors and defining a __get__ method, but nothing anywhere talks about calling a __get__method on another method. Is this undocumented behaviour that you just happen to be lucky enough to have stumbled across, or is there some specific logic as to why this works. I am loath to rely on undocumented behaviour in production code, so I would really appreciate it if you could explain why __get__ is working in this situation. Thanks!

  9. Mu Mind says:

    I was really confused by that, too. Apparently it’s using the “Descriptor Protocol” (http://docs.python.org/howto/descriptor.html#descriptor-protocol) on the function, or something like that, to turn it into a bound method. If that’s all it’s doing, though, I think this recipe is a lot clearer: http://countergram.com/adding-bound-methods.

    On another note, I don’t think the {“endpoint”: “http://…”} part has any effect (i.e., the code would do the same thing if you turned method_dictionary into a set or list of method names). The code kind of implies that those kwargs will get partial’ed in, but I don’t see how that would happen, and trying the code out, it does not in fact happen.

  10. Mu Mind says:

    Actually, I vote that this is silly. If you’re going to spec out each missing method anyway, you save nothing over just defining your methods and making them pass through to your stub implementation, like so: https://gist.github.com/3848795

    Am I missing something?

  11. Ryan says:

    Yeah, you can in fact just throw a stub in there, and it’s largely the same. Twython was originally built this way, but over time it got to be annoying to maintain. There’s something to be said for the simplicity of defining all endpoints in a dictionary and letting 99% of the rest be unchanging.

    Yes, there’s other ways to do it, but I thought this was a fun way. YMMV. :)

Leave a Reply