MODULE: Ingestion and schema matching

Many of the most illuminating discussions i have with people on any project is over diagrams.  When someone makes that box connect to that circle via an arrow (or better yet, a dotted line), that is usually when the penny drops for me.

Here are a few of the scribbles that facilitated conversation this week.

The diagram above was used for thinking through the critical path for ingestion new data into DAAO and how it impacts on schema design.

This is what this diagram currently means to me. Our designers and I will be working with CIs (you, imagined reader) to find out how an improved database website will best serve your research needs. Your answers input into modifying the schema design and also guide the design of the website through which you experience the database.  Meanwhile,  Olivia and her editorial/indexing team will mine print new source texts for ingestion – indexing to the new schema.  While Olivia (she’s busy!), the designers and I also go through the new databases, (such as art@base) that we are ingesting and interoperating with (Fuam), and matching and modifying when necessary  non-aligned schema. The next iteration of this diagram should have arrows all folding through these boxes in feedback loops – but the green pen died.

relational database twister

Ross and I were using this twister diagram to talk through how various filters might work to capture new imagined relationships across expanded ‘people’ set. – like artist-curator-designer/design organisation. (There is another re-conceptualisation of ‘organisations’ diagram coming)

This diagram emerged after a very interesting discussion with Mr Snow and Zina from the House of Laudanum – where we discussed metadata registry as way of thinking through ‘truth problem’ with interoperating databases – like Video Art Online (VAO) and DAAO.

There are so many things going in this project: ingesting, connecting, interoperating, reducing ambiguity, enhancing search, standardising, redesigning user experience AND then opening up our data into the miasma of the dataverse so it will be safe in numbers but simultaneously pulling it back again as something defined and redefined.

One of the most personally exciting things about this initial business analysis stage is  gaining a really solid sense of the dynamism and complexity of the landscape in which the DAAO needs to live if it is to survive.  There is so much bending and breaking of things to make things fit and yet there is a drift and flow. At once mechanical and organic, the dataverse is a fascinating place at the moment. New vocabularies emerge every other days to describe the growing, living, dead and dying worlds of data.

This week’s word is agile. (Well it been the word for a while) But as i try and work through critical paths of this project, it is clear that feedback across multiple processes makes linear process difficult. The need to be agile runs from product to process and back.