XML Walker section

An XML walker source section yields a hierarchy of items by iterating over an `lxml.etree`_ tree of XML elements that match an `XPath`_. This can be used to build content structure based on the sitemap or navigation of a HTML web site.

Options starting with element- may contain expressions whose value will be inserted into the element items. The expressions have access to the following:

element	the current walked element
item	the current walked element item to be yielded
source_item	the original item containing the walked tree
tree	the original walked tree
transmogrifier	the transmogrifier
name	the name of the inserter section
options	the inserter options
modules	sys.modules

Start with an HTML file containing a heirarchical navbar.

>>> import os
>>> html_file = os.path.join(
...     os.path.dirname(__file__), 'xmlwalker.html')

>>> infologger = """
... [transmogrifier]
... pipeline =
...     source
...     parse
...     walk
...     defaultpage
...     clean
...     logger
...
... [source]
... blueprint = collective.transmogrifier.sections.tests.rangesource
... size = 1
...
... [parse]
... blueprint = collective.transmogrifier.sections.inserter
... key = string:_trees
... value = python:modules['lxml.html'].parse('{}').xpath(\
...     "//*[contains(@class, 'navbar')]//ul[contains(@class, 'nav')]")
...
... [walk]
... blueprint = collective.transmogrifier.sections.xmlwalker
... element-keys =
...     _path
...     title
... element-_path = python:element.attrib.get(\
...     'href', element.attrib.get('src', ''))
... element-title = python:element.text_content().strip()\
...     or element.attrib.get('alt', '')
...
... [defaultpage]
... blueprint = collective.transmogrifier.sections.inserter
... key = string:_defaultpage
... condition = python:item.get('_parent', dict()).pop('_parent', True)\
...     and item.get('_defaultpage')
... value = exists:item/_defaultpage
...
... [clean]
... blueprint = collective.transmogrifier.sections.manipulator
... delete =
...     _trees
...     _element
...     id
...
... [logger]
... blueprint = collective.transmogrifier.sections.logger
... name = logger
... level = INFO
... """.format(html_file)
>>> registerConfig(u'collective.transmogrifier.sections.tests.xmlwalker',
...                infologger)
>>> transmogrifier(u'collective.transmogrifier.sections.tests.xmlwalker')
>>> print handler
logger INFO
  {}
logger INFO
  {'_parent': {}, '_path': '#', '_type': 'Folder', 'title': 'Foo Tab'}
logger INFO
    {'_is_defaultpage': True,
   '_parent': {'_path': '#', '_type': 'Folder', 'title': 'Foo Tab'},
   '_path': '#',
   'title': 'Foo Tab'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Foo Tab'},
   '_path': '../foo-tab/index.html',
   'title': 'Foo Tab Default Page'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Foo Tab'},
   '_path': '../foo-tab/bar-image.png',
   'title': 'Bar Image'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Foo Tab'},
   '_path': '../foo-tab/qux-page.html',
   'title': 'Qux Page'}
logger INFO
  {'_parent': {}, '_path': '#', '_type': 'Folder', 'title': 'Company'}
logger INFO
    {'_is_defaultpage': True,
   '_parent': {'_path': '#', '_type': 'Folder', 'title': 'Company'},
   '_path': '#',
   'title': 'Company'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Company'},
   '_path': '../company/news.html',
   '_type': 'Folder',
   'title': 'News'}
logger INFO
    {'_is_defaultpage': True,
   '_parent': {'_path': '../company/news.html',
               '_type': 'Folder',
               'title': 'News'},
   '_path': '../company/news.html',
   'title': 'News'}
logger INFO
    {'_parent': {'_path': '../company/news.html',
               '_type': 'Folder',
               'title': 'News'},
   '_path': '../company/news.html',
   'title': 'News'}
logger INFO
    {'_parent': {'_path': '../company/news.html',
               '_type': 'Folder',
               'title': 'News'},
   '_path': '../company/press_releases.html',
   'title': 'Press Releases'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Company'},
   '_path': '../company/events.html',
   'title': 'Events'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Company'},
   '_path': '../contact_us/contact.html',
   'title': 'Contact Us'}
logger INFO
    {'_parent': {'_path': '#', '_type': 'Folder', 'title': 'Company'},
   '_path': '../company/index.html',
   '_type': 'Folder',
   'title': 'About Company'}
logger INFO
    {'_is_defaultpage': True,
   '_parent': {'_path': '../company/index.html',
               '_type': 'Folder',
               'title': 'About Company'},
   '_path': '../company/index.html',
   'title': 'About Company'}
logger INFO
    {'_parent': {'_path': '../company/index.html',
               '_type': 'Folder',
               'title': 'About Company'},
   '_path': '../company/management.html',
   'title': 'Management'}
logger INFO
    {'_parent': {'_path': '../company/index.html',
               '_type': 'Folder',
               'title': 'About Company'},
   '_path': '../company/investors.html',
   'title': 'Investors'}
logger INFO
    {'_parent': {'_path': '../company/index.html',
               '_type': 'Folder',
               'title': 'About Company'},
   '_path': '../company/careers.html',
   'title': 'Careers'}
logger INFO
    {'_parent': {'_path': '../company/index.html',
               '_type': 'Folder',
               'title': 'About Company'},
   '_path': '../company/company.html',
   'title': 'About Us'}