View the Source Code

Run unit tests

Monomotapa - A Micro CMS
Copyright (C) 2014, Paul Munday.

PO Box 28228, Portland, OR, USA 97228
paul at

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero  Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
GNU General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <>.

See for licensing details.

    a city whose inhabitants are bounded by deep feelings of friendship,
    so thatthey intuit one another's most secret needs and desire. For
    instance, if one dreams that his friend is sad, the friend will perceive
    the distress and rush to the sleepers rescue.

    (Jean de La Fontaine, *Fables choisies, mises en vers*, VIII:11 Paris,
    2nd ed., 1678-9)

cited in :
Alberto Manguel and Gianni Guadalupi, *The Dictionary of Imaginary Places*,
Bloomsbury, London, 1999.

A micro app written using the Flask microframework to manage my personal site.
It is designed so that publishing a page requires no more than dropping a
markdown page in the appropriate directory (though you need to edit a json file
if you want it to appear in the top navigation).

It can also display its own source code and run its own unit tests.

The name 'monomotapa' was chosen more or less at random (it shares an initial
with me) as I didn't want to name it after the site and be typing import
paulmunday, or something similar, as that would be strange.

# pylint: disable=no-name-in-module, redefined-outer-name
from flask import render_template, abort, Markup, escape, request
from flask import make_response

from pygments import highlight
from pygments.lexers import PythonLexer, HtmlDjangoLexer, TextLexer
from pygments.formatters import HtmlFormatter

import markdown

import os.path
import os
import json
import random
import subprocess
import tempfile
from collections import OrderedDict

from monomotapa import app
from monomotapa.config import ConfigError

class MonomotapaError(Exception):
    """create classs for own errors"""

def get_page_attributes(jsonfile):
    """Returns dictionary of page_attributes.
    Defines additional static page attributes loaded from json file.
    N.B. static pages do not need to have attributes defined there,
    it is sufficient to have a in src for each /page
    possible values are src (name of markdown file to be rendered)
    heading, title, and trusted (i.e. allow embeded html in markdown)"""
        with open(src_file(jsonfile), 'r') as pagesfile:
            page_attributes = json.load(pagesfile)
    except IOError:
        page_attributes = []
    return page_attributes

def get_page_attribute(attr_src, page, attribute):
    """returns attribute of page if it exists, else None.
    attr_src = dictionary(from get_page_attributes)"""
    if page in attr_src and attribute in attr_src[page]:
        return attr_src[page][attribute]
        return None

# Navigation
def top_navigation(page):
    """Generates navigation as an OrderedDict from navigation.json.
    Navigation.json consists of a json array(list) "nav_order"
    containing the names of the top navigation elements and
    a json object(dict) called "nav_elements"
    if a page is to show up in the top navigation
    there must be an entry present in nav_order but there need not
    be one in nav_elements. However if there is the key must be the same.
    Possible values for nav_elements are link_text, url and urlfor
    The name  from nav_order will be used to set the link text,
    unless link_text is present in nav_elements.
    url and urlfor are optional, however if ommited the url wil be
    generated in the navigation by  url_for('staticpage', page=[key])
    equivalent to  @app.route"/page"; def page())
    which may not be correct. If a url is supplied  it will be used
    otherwise if urlfor is supplied it the url will be
    generated with url_for(urlfor). url takes precendence so it makes
    no sense to supply both.
    Web Sign-in is supported by adding a "rel": "me" attribute.
    with open(src_file('navigation.json'), 'r') as navfile:
        navigation = json.load(navfile)
    base_nav = OrderedDict({})
    for key in navigation["nav_order"]:
        nav = {}
        nav['base'] = key
        nav['link_text'] = key
        if key in navigation["nav_elements"]:
            elements = navigation["nav_elements"][key]
        base_nav[key] = nav
    return {'navigation':  base_nav, 'page': page}

# For pages
class Page(object):
    """Generates  pages as objects"""
    # pylint: disable=too-many-instance-attributes
    def __init__(self, page, **kwargs):
        """Define attributes for  pages (if present).
        Sets, self.title, self.heading, self.trusted etc
        This is done through indirection so we can update the defaults
        (defined  in the 'attributes' dictionary) with values from config.json
        or pages.json easily without lots of if else statements.
        If css is supplied it will overide any default css. To add additional
        style sheets on a per page basis specifiy them in pages.json.
        The same also applies with hlinks.
        css is used to set locally hosted stylesheets only. To specify
        external stylesheets use hlinks: in config.json for
        default values that will apply on all pages unless overidden, set here
        to override the default. Set in pages.json to add after default.
        # set default attributes = page.rstrip('/')
        self.defaults = get_page_attributes('defaults.json')
        self.pages = get_page_attributes('pages.json')
        title = page.lower()
        heading = page.capitalize()
            self.default_template = self.defaults['template']
        except KeyError:
            raise ConfigError('template not found in default.json')
        # will become, self.title, self.heading,
        # self.footer, self.internal_css, self.trusted
        attributes = {
            'name':, 'title': title,
            'navigation': top_navigation(,
            'heading': heading, 'footer': None,
            'css': None, 'hlinks': None, 'internal_css': None,
            'trusted': False
        # set from defaults
        # override with kwargs
        # override attributes if set in pages.json
        if page in self.pages:
        # set attributes (as etc)  using indirection
        for attribute, value in attributes.iteritems():
            setattr(self, attribute, value)
            setattr(self, attribute, value)
        # reset these as we want to append rather than overwrite if supplied
        if 'css' in kwargs:
            self.css = kwargs['css']
        elif 'css' in self.defaults:
            self.css = self.defaults['css']
        if 'hlinks' in kwargs:
            self.hlinks = kwargs['hlinks']
        elif 'hlinks' in self.defaults:
            self.hlinks = self.defaults['hlinks']
        # append hlinks and css from pages.json rather than overwriting
        # if css or hlinks are not supplied they are set to default
        if page in self.pages:
            if 'css' in self.pages[page]:
                self.css = self.css + self.pages[page]['css']
            if 'hlinks' in self.pages[page]:
                self.hlinks = self.hlinks + self.pages[page]['hlinks']
        # append heading to default if set in config
        if app.config['default_title']:
            self.title = app.config['default_title'] + self.title

    def _get_markdown(self):
        """returns rendered markdown or 404 if source does not exist"""
        # pylint: disable=no-member
        src = self.get_page_src(, 'src', 'md')
        if src is None:
            return render_markdown(src, self.trusted)

    def get_page_src(self, page, directory=None, ext=None):
        """"return path of file (used to generate page) if it exists,
        or return none.
        Also returns the template used to render that page, defaults
        to static.html.
        It will optionally add an extension, to allow
        specifiying pages by route."""
        # is it stored in a config
        pagename = get_page_attribute(self.pages, page, 'src')
        if not pagename:
            pagename = page + get_extension(ext)
        if os.path.exists(src_file(pagename, directory)):
            return src_file(pagename, directory)
            return None

    def get_template(self, page):
        """returns the template for the page"""
        pagetemplate = get_page_attribute(self.pages, page, 'template')
        if not pagetemplate:
            pagetemplate = self.default_template
        if os.path.exists(src_file(pagetemplate, 'templates')):
            return pagetemplate
            raise MonomotapaError("Template: %s not found" % pagetemplate)

    def generate_page(self, contents=None):
        """return a page generator function.
        For static pages written in Markdown under src/.
        contents are automatically rendered.
        N.B. See note above in about headers"""
        if not contents:
            contents = self._get_markdown()
        template = self.get_template(
        return render_template(

# helper functions

def src_file(name, directory=None):
    """return potential path to file in this app"""
    if not directory:
        return os.path.join('monomotapa', name)
        return os.path.join('monomotapa', directory, name)

def get_extension(ext):
    '''constructs extension, adding or stripping leading . as needed.
    Return null string for None'''
    if ext is None:
        return ''
    elif ext[0] == '.':
        return ext
        return '.%s' % ext

def render_markdown(srcfile, trusted=False):
    """Return markdown file rendered as html. Defaults to untrusted:
        html characters (and character entities) are escaped
        so will not be rendered. This departs from markdown spec
        which allows embedded html."""
        with open(srcfile, 'r') as f:
            src =
            src = src.decode('utf-8')
            if trusted is True:
                return markdown.markdown(src)
                return markdown.markdown(escape(src))
    except IOError:
        return None

def render_pygments(srcfile, lexer_type):
    """returns src(file) marked up with pygments"""
    if lexer_type == 'python':
        with open(srcfile, 'r') as f:
            src =
            contents = highlight(src, PythonLexer(), HtmlFormatter())
    elif lexer_type == 'html':
        with open(srcfile, 'r') as f:
            src =
            contents = highlight(src, HtmlDjangoLexer(), HtmlFormatter())
    # default to TextLexer for everything else
        with open(srcfile, 'r') as f:
            src =
            contents = highlight(src, TextLexer(), HtmlFormatter())
    return contents

def get_pygments_css(style=None):
    """returns css for pygments, use as internal_css"""
    if style is None:
        style = 'friendly'
    return HtmlFormatter(style=style).get_style_defs('.highlight')

def heading(text, level):
    """return as html heading at h[level]"""
    hltag = 'h{}'.format(str(level))
    return '\n<{}>{}</{}>\n'.format(hltag, text, hltag)

def name_obfuscator(name):
    """returns name plus a random six character string joined by a .
    characters are a..z or 0..9"""
    container = [name, '.']
    for _ in range(6):
        # 97 = asci a
        randint = random.randint(97, 132)
        # 122 = z, if over that  subtracting 75 converts it to  0..9
        if randint > 122:
            randint = randint - 75
        # add asci character
    return "".join(container)

# Define routes

def page_not_found(e):
    """ provides basic 404 page"""
    # pylint: disable=unused-argument
    defaults = get_page_attributes('defaults.json')
        css = defaults['css']
    except KeyError:
        css = None
    pages = get_page_attributes('pages.json')
    if '404' in pages:
        if'css' in pages['404']:
            css = pages['404']['css']
    return render_template(
        title="404::page not found",
        heading="Page Not Found",
        contents=Markup("This page is not there, try somewhere else.")
    ), 404

def index():
    """provides index page"""
    index = Page('index')
    return index.generate_page()

# default route is it doe not exist elsewhere
def staticpage(page):
    """ display a static page rendered from markdown in src
    i.e. displays /page or /page/ as long as src/ exists.
    srcfile, title and heading may be set in the pages global
    (ordered) dictionary but are not required"""
    static_page = Page(page)
    return static_page.generate_page()

# specialized pages
def catalogue_card(isbn):
    """Book specific pages"""
    catalogue_card = Page('catalogue_card', title=isbn, heading=isbn)
    page_src = catalogue_card.get_page_src(isbn, 'library', 'md')
    contents = render_markdown(page_src, True)
    return catalogue_card.generate_page(contents=contents)

def source():
    """Display source files used to render a page"""
    source_page = Page(
        'source', title="view the source code",
        heading="View the Source Code",
    page = request.args.get('page')
    # get source for markdown if any. 404's for non-existant markdown
    # unless special page eg source
    pagesrc = source_page.get_page_src(page, 'src', 'md')
    special_pages = ['source', 'unit-tests', '404']
    if page not in special_pages and pagesrc is None:
    # set enable_unit_tests  to true  in config.json to allow
    #  unit tests to be run  through the source page
    if app.config['enable_unit_tests']:
        contents = '''<p><a href="/unit-tests" class="button">Run unit tests
        # render if needed
        if page == 'unit-tests':
            contents += heading('', 2)
            contents += render_pygments('', 'python')
        contents = ''
    # render
    contents += heading('', 2)
    contents += render_pygments(
        source_page.get_page_src(''), 'python'
    # render markdown if present
    if pagesrc:
        contents += heading(os.path.basename(pagesrc), 2)
        contents += render_pygments(pagesrc, 'markdown')
    # render jinja templates
    contents += heading('base.html', 2)
    contents += render_pygments(
        source_page.get_page_src('base.html', 'templates'), 'html'
    template = source_page.get_template(page)
    contents += heading(template, 2)
    contents += render_pygments(
        source_page.get_page_src(template, 'templates'), 'html'
    return source_page.generate_page(contents)

def unit_tests():
    """display results of unit tests"""
    unittests = Page(
        'unit-tests', heading="Test Results",
    # exec unit tests in subprocess, capturing stderr
    capture = subprocess.Popen(
        ["python", ""],
        stdout=subprocess.PIPE, stderr=subprocess.PIPE
    output = capture.communicate()
    results = output[1]
    contents = '''<p>
    <a href="/unit-tests" class="button">Run unit tests</a>
    <div class="output" style="background-color:'''
    if 'OK' in results:
        color = "#ddffdd"
        result = "TESTS PASSED"
        color = "#ffaaaa"
        result = "TESTS FAILING"
    contents += ('''%s">\n<strong>%s</strong>\n<pre>%s</pre>\n</div>\n'''
                 % (color, result, results))
    # render
    contents += heading('', 2)
    contents += render_pygments('', 'python')
    return unittests.generate_page(contents)

def resume():
    """resume page with footer"""
    # pylint: disable=attribute-defined-outside-init
    resume = Page('resume')
    name = name_obfuscator('paul')
    footer_content = '''<span style="font-size: 1em">
    Download this resume as a <a href="/resume.pdf">PDF</a>.<br>
    Contact me: <script>address("%s","paulmunday",2,"")</script>''' % name
    resume.footer = Markup(footer_content)
    return resume.generate_page()

def resume_pdf():
    """renders up to date  resume as pdf from markdown"""
    # we need to use actual files here are pdf rendering is
    # done by call to external util
    tmpfile = tempfile.NamedTemporaryFile(delete=False)
    resume = src_file('', 'src')
    output = src_file('resume.pdf', 'src')
    name = name_obfuscator('paul')
    domain = ''
    header = "#Paul Munday\nwww.{}&nbsp;&nbsp;{}@{}\n\n".format(
        domain, name, domain)
    with open(, 'a') as f:
        with open(resume, 'r') as rfile:
            contents =
    # render pdf and remove temp file["pandoc",, "-S", "-o", output])
    # read pdf in as string and create flask response
    with open(output, 'rb') as pdfile:
        pdf =
    response = make_response(pdf, 200)
    response.headers["Content-Type"] = "Application/pdf"
    return response

## Key Skills and Technologies

Python, JavaScript, HTML/CSS, Bash/Shell Scripting, Perl, PHP, SQL, Processing, Git.

**Linux/Unix Systems and Network Administration**
Ubuntu, Debian, Red Hat. Apache, MySQL, LAMP stack,  Email(Postfix), DNS,
Salt Stack, Ansible , Fabric, Wordpress, Drupal, Firewalling with Iptables.
Network and security analysis with Wireshark, Nmap, tcpdump etc.
Linux-HA, DRBD, Heartbeat.

**Technical Support**
Linux specialist. Third Line technical support and department manager.

###  Other Skills
A certified and experienced tutor/trainer, I have devised and taught a number of IT
based courses and workshops including web design, introductory computing, and sound
editing and radio production.

**Management** I have extensive experience in the non-profit sector, managing paid
and volunteer staff. This has given me experience and knowledge of the particular
challenges faced in this sector including training, recruiting and managing
volunteers. It has also given me a wider set of skills in leadership, facilitation
and community building.

## Work Experience
**June 2016&ndash;Present: Lead Software Engineer, Cinderstaffing**

*October 2016&ndash;present*
I am currently working on a project to extend the use of the SEED platform to to the residential building sector. I continue to do open source work on SEED as well as work to develop a hosted SEED platform.

*June 2016&ndash;October 2016*
I worked as a full time open source Python/Django developer on the SEED Platform, an open source project, backed by the US Department of Energy, that enables local governments to aggregate and manage building energy performance data.

**May 2014&ndash;June 2016: Lead Software Engineer, Cinderstaffing**
I was employed as a full stack Python/Django developer, in an agile environment. I worked with Django  versions 1.5 and up,  as a member of a team working on a large product with over 30 million unique visitors a month and have also been the sole developer responsible for building an app from the ground up. I was promoted to Lead Software Engineer in January 2015.

*Additional Duties:* I was responsible for managing a team of 7+ engineers, working on 3 different projects. My duties include recruiting and interviewing for technical positions. I initiated a paid internship program to bring new developers, especially those from non-traditional backgrounds, to a point where they can be employed as full time engineers. As part of this we developed and successfully launched  a new scheduling web app for small businesses that allows them to transition from paper based systems. My responsibilities included providing project management and software architecture for this as well as mentoring the interns.

**September 2013&ndash;June 2104: IT Consultant, Northwest Workers Justice Project**
Donated time as IT Consultant to a Portland based non-profit on a ground up
rebuild of their server infrastructure.

- Developed configuration management using Salt Stack.
- Implemented replacement for hardware setup using virtual machines built on
QEMU/KVM/Libvirt and Debian GNU/Linux.
- Wrote custom code in Python to transition data off legacy software.
- Created open source Python module for interacting with CiviCRM REST API.

**November 2010&ndash;December 2013: Technical Support Specialist/Coordinator, Free Geek**
Free Geek is a non-profit that donates 1000 Ubuntu  machines a year to
non-profits and others with free tech support.

- Managed tech support department providing hardware and software support.
- Provided third line support (Ubuntu Linux).
- Created a suite of tools for automating tech support tasks including a tool
 for automatic backup and system and data replication across different hardware. (Bash/shell).
- Wrote a Python library to interact with ticket system api for data reporting
- Developed data recovery process.
- Implemented use of electronic ticket system, replacing paper system.
- Developed and documented tech support procedures.
- Linux system administration (Ubuntu and Debian GNU/Linux. Apache, MySQL, RT,
- Joint manager of organization.

** April 2010&ndash;November 2010: Tutor, SUN project Ron Russell Middle School**
Worked as part time tutor at an after school project teaching IT related courses
(e.g. web design).

*February 2009&ndash;May 2010: Relocated to the USA. Worked as a volunteer
(due to contractual obligations and visa status) in a number of roles including
IT consultant and teacher.*

** Sept 2005&ndash;January 2009: Systems Administrator, Red Snapper Ltd. **
Full service web development agency.

- Managed multiple servers providing hosting for clients using Apache, MySQL
and in house CMS.
- Implemented replacement high availability cluster using Linux-HA, Heatbeat
and DRBD for full mirroring across all servers for > 99.9% uptime and <5s
failover times.
- Provided DNS and Email using Postfix/MySQL and TinyDNS for clients and
internal use.
- Managed and implemented internal services such  as vsftpd, LDAP, SVN, IP tables firewall  etc.
- Sole responsibility for Linux systems administration (Debian).
- Worked with developers on managing internal software releases.
- HTML/CSS coding.

** September 2003&ndash;September 2005: Tutor, Bristol Wireless Co-operative**
Taught courses, did design work and assisted systems administration for one of
Europe's largest non-profit wireless networks using Linux.

** June 2000&ndash;December 2002: Systems Administrator, 2Bet Ltd.**
Responsible for all systems and network administration for a software development
company running Tomcat, Apache and MySQL (Linux/Solaris).

** June 1998&ndash;June 2000: IT Development Worker, BPEC**
Set up the UK's first low cost public Internet access facility for an educational
and environmental non-profit using Linux server with Windows clients..

** September 1996&ndash;May 1998: IT Coordinator, River Ocean Research and Education**
Responsible for IT for an environmental non-profit (Linux/Windows).

## Professional Affiliations
ACM Member (Association for Computing Machinery). 

## Community Service and Artistic Practice
**IT &amp; Open Source:** I donate time to non-profits as an IT
consultant and I am a contributor to, and author of, open source projects.

**Artistic Practice**
My artistic practise art is based around using the web as technology for conceptual art.  My current focus is on (producing software for) mapping as a tool for digital storytelling and the exploration of the invisble and the intagible in the world around us.

**Board Service.**
I have served as a board member for a number of co-ops and non-profits.

I trained for three years in Dance and Performance, and have performed at a
semi-professional level.

**Radio production and presentation**
I have five years experience of working in community radio.

**Graphic design**
I have done graphic design work on a voluntary, and occasionally professional,
basis for many years.

Volunteer mechanic in community bike projects. Volunteer at Repair Cafe.


<!DOCTYPE html>

Monomotapa - A Micro CMS
Copyright (C) 2014, Paul Munday.

PO Box 28228, Portland, OR, USA 97228
paul at

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero  Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
GNU General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <>.

There should also be a copy of the AGPL in src/ that should be
accessible by going to <a href ="/license">/license<a> on this site.
<title>{% if title -%}{{title}}{% endif %}</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
{%- if css -%}
{%- for file in css %}
    <link href="{{ url_for('static', filename=file) }}" rel="stylesheet" type="text/css"  />
    {%- endfor -%}
{%- endif %}
{% if internal_css %}
<style type="text/css">
{% endif %}
{%- if hlinks -%}
{%- for item in hlinks -%}
        {%- if item.href %} href="{{item.href}}"{% endif -%}
        {%- if item.rel %} rel="{{item.rel}}"{% endif -%} 
        {%- if item.type %} type="{{item.type}}"{% endif -%}
        {%- if %} type="{{}}"{% endif -%}
        {%- if item.hreflang %} type="{{item.hreflang}}"{% endif -%}
        {%- if item.charset %} type="{{item.charset}}"{% endif -%}
{% endfor %}
{%- endif -%}
<div id="wrap">
    <div id="main">
        <div id="nav">
           <ul id="nav">
            {%- for item in navigation.navigation.itervalues() -%}
                <li><a href="
                {%- if item.url -%}{{item.url}}
                {%- elif item.urlfor -%}
                    {%- if item.urlfor == "source" -%}
                        {{ url_for(item.urlfor, }}
                    {%- else -%}
                        {{ url_for(item.urlfor) }}
                    {%- endif -%}
                {%- else -%}
                    {{ url_for('staticpage', page=item.base) }}
                {%- endif -%}
                {%- if item.rel -%}
                         " rel="{{item.rel}} 
                    {%- endif -%}
                {% endfor -%}
        {% block content %}{% endblock %}
<div id="footer">
    <p id="footer">
    <!-- footer goes here -->
    {% if footer %}
    {% endif %}



<!DOCTYPE html>
<link href='' rel='stylesheet' type='text/css'>
<title>{% if title -%}{{title}}{% endif %}</title>
<link href="{{ url_for('static', filename='style.css') }}" rel="stylesheet" type="text/css"  />
{% if css %}
    {% for file in css %}
        <link href="{{file}}" rel="stylesheet" type="text/css"  />
    {% endfor %}
{% endif %}
{% if internal_css %}
<style type="text/css">
{% endif %}
<script language="javascript" src="/static/email.js"></script>
<div id="wrap">
    <div id="main">
        <div id="nav">
           <ul id="nav">
            {%- for item in navigation.navigation.itervalues() -%}
                <li><a href="
                {%- if item.url -%}{{item.url}}
                {%- elif item.urlfor -%}
                    {%- if item.urlfor == "source" -%}
                        {{ url_for(item.urlfor, }}
                    {%- else -%}
                        {{ url_for(item.urlfor) }}
                    {%- endif -%}
                {%- else -%}
                    {{ url_for('staticpage', page=item.base) }}
                {%- endif -%}
                {% endfor -%}
            {% if heading %}<h1>{{heading}}</h1>{% endif %}
        <div id="messages">
            {% with messages = get_flashed_messages() %}
                {% if messages %}
                    {% for message in messages %}
                        {{ message }}
                    {% endfor %}
                {% endif %}
            {% endwith %}
        <div id="content">
<div id="footer">
    <p class="footer">
    <!-- footer goes here -->
    {% if footer %}
    {% endif %}