| Comment Example |
| =============== |
| |
| .. contents:: |
| |
| Introduction |
| ------------ |
| |
| This is an example of how to write WSGI middleware with WebOb. The |
| specific example adds a simple comment form to HTML web pages; any |
| page served through the middleware that is HTML gets a comment form |
| added to it, and shows any existing comments. |
| |
| Code |
| ---- |
| |
| The finished code for this is available in |
| `docs/comment-example-code/example.py |
| <https://github.com/Pylons/webob/blob/master/docs/comment-example-code/example.py>`_ |
| -- you can run that file as a script to try it out. |
| |
| Instantiating Middleware |
| ------------------------ |
| |
| Middleware of any complexity at all is usually best created as a |
| class with its configuration as arguments to that class. |
| |
| Every middleware needs an application (``app``) that it wraps. This |
| middleware also needs a location to store the comments; we'll put them |
| all in a single directory. |
| |
| .. code-block:: python |
| |
| import os |
| |
| class Commenter(object): |
| def __init__(self, app, storage_dir): |
| self.app = app |
| self.storage_dir = storage_dir |
| if not os.path.exists(storage_dir): |
| os.makedirs(storage_dir) |
| |
| When you use this middleware, you'll use it like: |
| |
| .. code-block:: python |
| |
| app = ... make the application ... |
| app = Commenter(app, storage_dir='./comments') |
| |
| For our application we'll use a simple static file server that is |
| included with `Paste <http://pythonpaste.org>`_ (use ``easy_install |
| Paste`` to install this). The setup is all at the bottom of |
| ``example.py``, and looks like this: |
| |
| .. code-block:: python |
| |
| if __name__ == '__main__': |
| import optparse |
| parser = optparse.OptionParser( |
| usage='%prog --port=PORT BASE_DIRECTORY' |
| ) |
| parser.add_option( |
| '-p', '--port', |
| default='8080', |
| dest='port', |
| type='int', |
| help='Port to serve on (default 8080)') |
| parser.add_option( |
| '--comment-data', |
| default='./comments', |
| dest='comment_data', |
| help='Place to put comment data into (default ./comments/)') |
| options, args = parser.parse_args() |
| if not args: |
| parser.error('You must give a BASE_DIRECTORY') |
| base_dir = args[0] |
| from paste.urlparser import StaticURLParser |
| app = StaticURLParser(base_dir) |
| app = Commenter(app, options.comment_data) |
| from wsgiref.simple_server import make_server |
| httpd = make_server('localhost', options.port, app) |
| print 'Serving on http://localhost:%s' % options.port |
| try: |
| httpd.serve_forever() |
| except KeyboardInterrupt: |
| print '^C' |
| |
| I won't explain it here, but basically it takes some options, creates |
| an application that serves static files |
| (``StaticURLParser(base_dir)``), wraps it with ``Commenter(app, |
| options.comment_data)`` then serves that. |
| |
| The Middleware |
| -------------- |
| |
| While we've created the class structure for the middleware, it doesn't |
| actually do anything. Here's a kind of minimal version of the |
| middleware (using WebOb): |
| |
| .. code-block:: python |
| |
| from webob import Request |
| |
| class Commenter(object): |
| |
| def __init__(self, app, storage_dir): |
| self.app = app |
| self.storage_dir = storage_dir |
| if not os.path.exists(storage_dir): |
| os.makedirs(storage_dir) |
| |
| def __call__(self, environ, start_response): |
| req = Request(environ) |
| resp = req.get_response(self.app) |
| return resp(environ, start_response) |
| |
| This doesn't modify the response it any way. You could write it like |
| this without WebOb: |
| |
| .. code-block:: python |
| |
| class Commenter(object): |
| ... |
| def __call__(self, environ, start_response): |
| return self.app(environ, start_response) |
| |
| But it won't be as convenient later. First, lets create a little bit |
| of infrastructure for our middleware. We need to save and load |
| per-url data (the comments themselves). We'll keep them in pickles, |
| where each url has a pickle named after the url (but double-quoted, so |
| ``http://localhost:8080/index.html`` becomes |
| ``http%3A%2F%2Flocalhost%3A8080%2Findex.html``). |
| |
| .. code-block:: python |
| |
| from cPickle import load, dump |
| |
| class Commenter(object): |
| ... |
| |
| def get_data(self, url): |
| filename = self.url_filename(url) |
| if not os.path.exists(filename): |
| return [] |
| else: |
| f = open(filename, 'rb') |
| data = load(f) |
| f.close() |
| return data |
| |
| def save_data(self, url, data): |
| filename = self.url_filename(url) |
| f = open(filename, 'wb') |
| dump(data, f) |
| f.close() |
| |
| def url_filename(self, url): |
| # Double-quoting makes the filename safe |
| return os.path.join(self.storage_dir, urllib.quote(url, '')) |
| |
| You can get the full request URL with ``req.url``, so to get the |
| comment data with these methods you do ``data = |
| self.get_data(req.url)``. |
| |
| Now we'll update the ``__call__`` method to filter *some* responses, |
| and get the comment data for those. We don't want to change responses |
| that were error responses (anything but ``200``), nor do we want to |
| filter responses that aren't HTML. So we get: |
| |
| .. code-block:: python |
| |
| class Commenter(object): |
| ... |
| |
| def __call__(self, environ, start_response): |
| req = Request(environ) |
| resp = req.get_response(self.app) |
| if resp.content_type != 'text/html' or resp.status_code != 200: |
| return resp(environ, start_response) |
| data = self.get_data(req.url) |
| ... do stuff with data, update resp ... |
| return resp(environ, start_response) |
| |
| So far we're punting on actually adding the comments to the page. We |
| also haven't defined what ``data`` will hold. Let's say it's a list |
| of dictionaries, where each dictionary looks like ``{'name': 'John |
| Doe', 'homepage': 'http://blog.johndoe.com', 'comments': 'Great |
| site!'}``. |
| |
| We'll also need a simple method to add stuff to the page. We'll use a |
| regular expression to find the end of the page and put text in: |
| |
| .. code-block:: python |
| |
| import re |
| |
| class Commenter(object): |
| ... |
| |
| _end_body_re = re.compile(r'</body.*?>', re.I|re.S) |
| |
| def add_to_end(self, html, extra_html): |
| """ |
| Adds extra_html to the end of the html page (before </body>) |
| """ |
| match = self._end_body_re.search(html) |
| if not match: |
| return html + extra_html |
| else: |
| return html[:match.start()] + extra_html + html[match.start():] |
| |
| And then we'll use it like: |
| |
| .. code-block:: python |
| |
| data = self.get_data(req.url) |
| body = resp.body |
| body = self.add_to_end(body, self.format_comments(data)) |
| resp.body = body |
| return resp(environ, start_response) |
| |
| We get the body, update it, and put it back in the response. This |
| also updates ``Content-Length``. Then we define: |
| |
| .. code-block:: python |
| |
| from webob import html_escape |
| |
| class Commenter(object): |
| ... |
| |
| def format_comments(self, comments): |
| if not comments: |
| return '' |
| text = [] |
| text.append('<hr>') |
| text.append('<h2><a name="comment-area"></a>Comments (%s):</h2>' % len(comments)) |
| for comment in comments: |
| text.append('<h3><a href="%s">%s</a> at %s:</h3>' % ( |
| html_escape(comment['homepage']), html_escape(comment['name']), |
| time.strftime('%c', comment['time']))) |
| # Susceptible to XSS attacks!: |
| text.append(comment['comments']) |
| return ''.join(text) |
| |
| We put in a header (with an anchor we'll use later), and a section for |
| each comment. Note that ``html_escape`` is the same as ``cgi.escape`` |
| and just turns ``&`` into ``&``, etc. |
| |
| Because we put in some text without quoting it is susceptible to a |
| `Cross-Site Scripting |
| <http://en.wikipedia.org/wiki/Cross-site_scripting>`_ attack. Fixing |
| that is beyond the scope of this tutorial; you could quote it or clean |
| it with something like `lxml.html.clean |
| <http://codespeak.net/lxml/lxmlhtml.html#cleaning-up-html>`_. |
| |
| Accepting Comments |
| ------------------ |
| |
| All of those pieces *display* comments, but still no one can actually |
| make comments. To handle this we'll take a little piece of the URL |
| space for our own, everything under ``/.comments``, so when someone |
| POSTs there it will add a comment. |
| |
| When the request comes in there are two parts to the path: |
| ``SCRIPT_NAME`` and ``PATH_INFO``. Everything in ``SCRIPT_NAME`` has |
| already been parsed, and everything in ``PATH_INFO`` has yet to be |
| parsed. That means that the URL *without* ``PATH_INFO`` is the path |
| to the middleware; we can intercept anything else below |
| ``SCRIPT_NAME`` but nothing above it. The name for the URL without |
| ``PATH_INFO`` is ``req.application_url``. We have to capture it early |
| to make sure it doesn't change (since the WSGI application we are |
| wrapping may update ``SCRIPT_NAME`` and ``PATH_INFO``). |
| |
| So here's what this all looks like: |
| |
| .. code-block:: python |
| |
| class Commenter(object): |
| ... |
| |
| def __call__(self, environ, start_response): |
| req = Request(environ) |
| if req.path_info_peek() == '.comments': |
| return self.process_comment(req)(environ, start_response) |
| # This is the base path of *this* middleware: |
| base_url = req.application_url |
| resp = req.get_response(self.app) |
| if resp.content_type != 'text/html' or resp.status_code != 200: |
| # Not an HTML response, we don't want to |
| # do anything to it |
| return resp(environ, start_response) |
| # Make sure the content isn't gzipped: |
| resp.decode_content() |
| comments = self.get_data(req.url) |
| body = resp.body |
| body = self.add_to_end(body, self.format_comments(comments)) |
| body = self.add_to_end(body, self.submit_form(base_url, req)) |
| resp.body = body |
| return resp(environ, start_response) |
| |
| ``base_url`` is the path where the middleware is located (if you run |
| the example server, it will be ``http://localhost:PORT/``). We use |
| ``req.path_info_peek()`` to look at the next segment of the URL -- |
| what comes after base_url. If it is ``.comments`` then we handle it |
| internally and don't pass the request on. |
| |
| We also put in a little guard, ``resp.decode_content()`` in case the |
| application returns a gzipped response. |
| |
| Then we get the data, add the comments, add the *form* to make new |
| comments, and return the result. |
| |
| submit_form |
| ~~~~~~~~~~~ |
| |
| Here's what the form looks like: |
| |
| .. code-block:: python |
| |
| class Commenter(object): |
| ... |
| |
| def submit_form(self, base_path, req): |
| return '''<h2>Leave a comment:</h2> |
| <form action="%s/.comments" method="POST"> |
| <input type="hidden" name="url" value="%s"> |
| <table width="100%%"> |
| <tr><td>Name:</td> |
| <td><input type="text" name="name" style="width: 100%%"></td></tr> |
| <tr><td>URL:</td> |
| <td><input type="text" name="homepage" style="width: 100%%"></td></tr> |
| </table> |
| Comments:<br> |
| <textarea name="comments" rows=10 style="width: 100%%"></textarea><br> |
| <input type="submit" value="Submit comment"> |
| </form> |
| ''' % (base_path, html_escape(req.url)) |
| |
| Nothing too exciting. It submits a form with the keys ``url`` (the |
| URL being commented on), ``name``, ``homepage``, and ``comments``. |
| |
| process_comment |
| ~~~~~~~~~~~~~~~ |
| |
| If you look at the method call, what we do is call the method then |
| treat the result as a WSGI application: |
| |
| .. code-block:: python |
| |
| return self.process_comment(req)(environ, start_response) |
| |
| You could write this as: |
| |
| .. code-block:: python |
| |
| response = self.process_comment(req) |
| return response(environ, start_response) |
| |
| A common pattern in WSGI middleware that *doesn't* use WebOb is to |
| just do: |
| |
| .. code-block:: python |
| |
| return self.process_comment(environ, start_response) |
| |
| But the WebOb style makes it easier to modify the response if you want |
| to; modifying a traditional WSGI response/application output requires |
| changing your logic flow considerably. |
| |
| Here's the actual processing code: |
| |
| .. code-block:: python |
| |
| from webob import exc |
| from webob import Response |
| |
| class Commenter(object): |
| ... |
| |
| def process_comment(self, req): |
| try: |
| url = req.params['url'] |
| name = req.params['name'] |
| homepage = req.params['homepage'] |
| comments = req.params['comments'] |
| except KeyError, e: |
| resp = exc.HTTPBadRequest('Missing parameter: %s' % e) |
| return resp |
| data = self.get_data(url) |
| data.append(dict( |
| name=name, |
| homepage=homepage, |
| comments=comments, |
| time=time.gmtime())) |
| self.save_data(url, data) |
| resp = exc.HTTPSeeOther(location=url+'#comment-area') |
| return resp |
| |
| We either give a Bad Request response (if the form submission is |
| somehow malformed), or a redirect back to the original page. |
| |
| The classes in ``webob.exc`` (like ``HTTPBadRequest`` and |
| ``HTTPSeeOther``) are Response subclasses that can be used to quickly |
| create responses for these non-200 cases where the response body |
| usually doesn't matter much. |
| |
| Conclusion |
| ---------- |
| |
| This shows how to make response modifying middleware, which is |
| probably the most difficult kind of middleware to write with WSGI -- |
| modifying the request is quite simple in comparison, as you simply |
| update ``environ``. |
