The Annotator
Suppose you are an historian, studying old sketches, and you have one
digitised as a bitmap picture. (Or, you might be a space scientist,
studying the recent photographs of the surface of one of Jupiter's moons.)
You want to annotate the picture, drawing shapes around significant
features and attaching meaning to them. Annotator is designed to help you
do this.
Here's a simple example. Note the picture is monochrome while the
annotation marks are in colour. Some are rectangles or polygons, showing
various items of interest; some are lines showing relationships. The
Annotator makes use of the Amiga's Dual PlayField graphics mode to keep the
annotations separate from the picture - as though you were drawing on a
sheet of glass covering the picture - so keeping the picture clean.
Annotator is not just a drawing program. It is not just a database. The
special thing about Annotator is that it links drawn marks with a knowledge
base of what those marks mean. It links
Moreover, it is designed to be easier to use than many drawing packages and
to employ a more powerful, more flexible data model than many databases.
Here is a brief list of some of its features:
- Ultra-direct drawing and modification - little use of toolbars while
drawing - so that your flow of thinking is not interrupted. A 'proximal user
interface'.
- Each graphical entity links to real semantics:
- Items. These are main things you wish to mark.
- Attributes. All items have attributes (of your
choice) e.g. description, date, what you infer from it.
- Relationships. You can link things together.
- Scenes. Special items that represent scenes in
your picture. Other items are automatically linked to scenes.
- Create your own types of item, attribute and relationship as needed.
When you need them.
- Generate text output from your items.
- Topics. Link your items and relationships to
topics such as 'death', 'money', 'love', 'beauty' - you can define the
topics of your choice.
- Lists. Form sets or lists of items and output
them.
- Annotator is *free*; download from (to be supplied).
1. (Installing Annotator: Download Annotator from (notyet). Then
decrunch using LHA. Then install it by double-clicking on the Install-
Annotator icon.)
2. Start Annotator by double-clicking on its icon, in the normal way.
You are presented with an
initial control screen.
3a. New picture? (To get existing one, see below.) Hit the 'New'
button and you will be asked which picture you wish to annotate. Select
one. A few examples have been included with the program. Your picture
should normally be monochrome or few colours for best results.
3b. Existing picture? Hit the 'Get KB' button. (KB stands for
'knowledge base', since you are storing your knowledge and beliefs and
interpretations along with the picture.) You will be asked for your KB
file. A simple one is supplied -
Execution, which shows an execution in progress with a few
annotations.
4. Now start drawing, using the mouse, as follows ..
- Most drawing done with mouse and left button (LMB).
- When you draw a new
item you will be immediately asked for its
name, meaning and attributes.
- Press LMB top-left of thing you wish to mark, and drag mouse down to
bottom-right - as normal practice in drawing. A simple rectangle.
- For a polygon, start as for rectangle but hit the space bar to set down
the second vertex, then keep dragging and hitting space for each vertex you
want to draw. You'll soon get used to it.
- (The following three use Qualifier Keys to obtain new operations ...)
- To
move a thing bodily, hold Left-Shift down (remember:
'Shift' key to shift a thing!), place mouse over an edge, line or vertex
of the thing, press LMB and drag to new position. Release when in right
place.
- To
draw a
relationship, hold Right-Alt key down as
you start drawing (remember: 'Alt' to draw alternative thing and 'R'
('right') for 'Relationship'). To draw a relationship you must start
drawing on the edge of an item and release over the edge of another.
- To
draw a
scene, hold Left-Alt key down as you start
drawing (remember: 'Alt' to draw alternative thing and 'L' ('left') could
stand for 'Locality').
- (To convert rectangle into polygon, start moving it as below and hit
'P' key - for 'polygon' - and release LMB. Thereafter it will behave as a
polygon, though of course at first it appears to have only 4 sides.)
- To modify a drawn item, simply place mouse on its edge, press LMB and
pull. Where the mouse is when you press LMB determines what modification
is done:
- On edge of rectangle: pull the edge to resize it.
- On corner of rectangle: pull two edges to resize.
- On line of polygon: make a bend - a new vertex.
- On vertex of polygon: move the vertex.
- When moving a vertex, hit Del key (next Help key) to remove vertex.
- To bring up semantic details, click (not press) mouse over the line of
a thing with right mouse button. See
Semantics.
Your drawn marks are more than marks: they mean something. In particular,
each thing usually has a name and/or a meaning. In addition there can be
any number of attributes, and relationships to other things.
Items are individual things, or objects, that you wish to pick out on
your picture, such as:*
- a person, a tool, a horse, a hand wielding a bloody sword in an
historical print,
- a mountain, an impact crater, a volcano, in a picture of Jupiter's
moon.
In drawing an item you are separating it out from its background (the
picture) because it is meaningful to you.
Most items are expressed by coloured rectangles or polygons on top of the
picture. To draw an item, simply press left mouse button and draw a
rectangle in the time-honoured way. Then an Attributes Panel comes up into
which you can fill:
- the name of the item
- a longer explanation of its meaning
- the values of any
attributes it has, and the standard ones in this
version are:
- Description (a string value)
- Date (a number)
- What you infer from the item (a string)
- Any cross referencing you wish to do (a string)
There are various types of item, and you can create new types as needed.
For a new KB (new picture) you are given the following types for free:
- Person
- Object
- Piece/Part (of another item)
- Clothing
- Tool
(though this list might change with new versions). They are shown at the
start of the Item Type List in the
KB Panel.
(The other item types are special for the system's use; they have
been left in view so you can gain some idea of them but I don't *think* you
will come to any grief if you draw them; it's just that they won't be very
useful to you at present.)
At any time you can bring up the item's
Item Details Panel which presents all the details of the
item, including its name, its colour and line pattern, its attributes and
all things it is related to. And you can alter many of these, such as
adding new attributes to it.
Attributes belong to items (and scenes) and hold
values that describe your items. Think of items as
nouns, and attributes and their values as adjectives. Attributes hold the
'data' of your items, for instance that the signified hanging occurred in
1471 and from this you might infer that it was part of the violent backlash
against the peasant revolt.
By standard, each item starts with four attributes:
- Description (a string value)
- Date (a number)
- What you infer from the item (a string)
- Any cross referencing you wish to do (a string)
and you are invited to enter the values of these when the item is first
drawn.
Note, the Name and Meaning of an item are not true 'attributes',
though you enter them at the same time; they cannot be got rid of though
they can be made blank.
You can enter their values, or change their values, later on via the
Item Details Panel - by selecting the attribute and
hitting the nearby 'See' button, which brings up the Attributes Details
Panel.
You can also add new attributes to the item (and get rid of unwanted ones)
via the
Item Details Panel - entering a name, selecting a type
and hitting the 'Add' button. Note that you can do this at any time and
are not, as in most databased systems, restricted to doing it at creation
time. For instance, you might want to add something about the style in
which the item was drawn.
To create a new Attribute Type hit the Att Types button on the
KB panel and up comes the Attributes Types Panel. On
the left hand side, enter a Name, select a Type of
value for it and hit the Add button. The right-hand
list is of names given to
enumeration or ordinal types.
Relationships are used to link items together in a way that carries
meaning for you. Most relationships are expressed as lines between items.
The main relationship you will use will be the
Normal Link.
To draw a relationship, simply:
- Press Right Alt button and hold it down.
- Place the mouse over a
line of an item (in the middle of it won't do).
- Drag until end of line is over line of another item.
- Release. And a line should be drawn.
(
Oops. No line is drawn? Normally this means you haven't
released over another item's line. Try again. It doesn't start drawing a
line but rather draws a rectangle instead? It's drawing another item,
probably because you have not pressed the mouse exactly over an item. Try
again.)
Many pictures, especially of the more complex sort found in historical
drawings, contain a number of scenes. Different things are happening in
different parts. It is useful to link all items in that area into a scene
- or into more than one.
You can draw a scene - usually a large green/red dashed rectangle - quite
easily, just as you would draw an item. After that, any items that lie
partly within that scene are linked to the scene by the 'Contains'
relationship. There are two ways of drawing a scene:
- Either select Scene as the Item Type on the
kbpanel.
- Or hold the Left Alt button down as you start drawing, and - whichever
item type is selected - it will draw a scene for you. That can be quite
convenient.
The use of scenes is up to you, but they give a visual grouping and an
automatic linking (unlike
Topics, which you have to set up and link specially.
More importantly, when
exporting your annotation scenes are recognised as
special entities.
In most annotations you will have a number of significant topics - such
as death, love, money - by which you might want to index your
items and
scenes. The Topics facility of Annotator allows you
to:
- Define your own topics as you like.
- Easily link items to topics.
- Highlight all items or scenes that relate to chosen topics.
There are two ways to
define Topics. One way, not really recommended,
is to select the Topics Item Type and draw boxes representing them
on the picture. Not recommended, cos it clutters up the picture, but you
might want to do it for some reason. The recommended way is to hit
the Topics button on the
KB Panel to bring up the Topics Control Panel. Then you
enter the name for the topic and hit the 'Add' button. You can add
several in this way and then either leave the panel up or hit 'Done'
button. Note that topics created in this way have no visual expression on
the picture.
There are two ways to
link items (and scenes) to topics. One way, not really
recommended,
and only relevant to topics drawn onto the picture, is to select the
Topic Relationship Type and draw a relationship from the topic box to the
item box. Not recommended cos if you draw the relationship in the reverse
direction (from item to topic) then the link is not recognised as a true
Topic link - and it is very easy to forget in which direction to draw
it. The recommended way it to bring up the item's (or scene's)
Item Details Panel and hit the Topics button you will
find there. This presents a list of topics and checkboxes which you can
hit to link or unlink the item to topics.
To
highlight a topic bring up the Topics Control Panel by
hitting the Topics button on the
KB Panel. On this, hit the 'Hide All' button to erase
all visible things from the picture (don't worry; they are still there,
just invisible). Then select a topic in the list and hit the 'Show'
button. All items linked to that topic will then be displayed. Select
another topic and hit 'Show' and these become displayed too.
To bring back all the items, hit the 'Sh' button next to the 'Curr Pic'
button on the
KB Panel.
When you are analysing things you often want them in groups, not one by
one. You might want to see all the persons who are kneeling, because those
people are likely to be paying homage. Or all who are gazing at people who
are kneeling. The lists feature helps you do this.
The Lists facility allows you to create sets or lists of items etc. that
are meaningful and then display them, export them, etc. For instance, you
can create a set of all items later than a particular date. To see it, hit
the Lists button on your KB Panel.
Up comes a rather complex panel, the Lists Panel. Don't be frightened:
it's in three main parts:
- The top line, which allows you to deal with one list at a time, to
select a list of interest and show its members. You can also export
details of them.
But how do you create a list? This is what the rest of the panel is about.
- The top half, in which you specify the source of your list, such
as "from all items", and
- The bottom half, in which you specify the selection criteria.
Then, when you hit the 'DoIt' button, Annotator will go through all things
in the source and apply the selection criteria to each, and all those
chosen will be placed in a new list. (For those who know databases, it is
a bit like the standard SQL SELECT .. FROM .. WHERE ..
statement, but Annotator allows you to do joins and counts a bit more
easily.)
The source can be one of:
- Search through all items, or all relationships
or both. Normally just the items. Normally you would only search user
types, but you can also search system types such as all topics, or even all
lists.
- Search through all items of a given type. Hit
the wee button to specify the Type.
- Search through all relationships of a given
type. Hit the wee button to specify the Type.
- Search through all things in one of your existing
lists, maybe to refine your search further. Hit the wee button to
specify what list.
- Search through all things related to things in that
list, such as all people gazing at them. (This is rather like doing a
relational join.) You must specify the relationship type ('None' means
'any'!), and the direction of the relationship: forward or backward or
both. (e.g. if you have a list of people and a 'gazing' relationship type,
do you want all things they are gazing at, or all people gazing at them?)
(If you specify no relationship type (i.e. to obtain all/any), then this
usually means only the user types, but can also include the system ones
too.)
Now that you have specified your source, you must say whether you want all
the things found or to select from them:
- First, the long button allows you to:
- take all of them, ignoring all selection criteria,
- take those that satisfy any of the criteria, or
- take all except those that satisfy any of them.
- First selection criterion is those things that contain a certain small
string of letters, and you can specify whether this is in their name, their
meaning or in any string attribute. If you want this selection criterion
active, hit the button to the left so it says 'Those that' rather than
'(Ignore this)'.
- You can also select on attributes of other types, though this
facility is not working fully at present 27 September 1997. You select
the name of a domain, and what you want about the domain: just that the
thing has an attribute of this type, or that its value is compared with
some given value.
- You can select on topology: connectivity. Those things that have at
least a certain number of relationships either to or from other things.
Those things that are more connected are likely to be the more important
things in your annotation. (At present, it counts system as well as user
relationships, so if a thing is in many lists, it will have many
antecedents.)
At the bottom of the panel is the 'DoIt' button which, when you hit it,
makes the new list. First thing that happens is an Item Details Panel
comes up for the 'new list'. You should rename it as needed and hit 'OK'.
Then at the top of the panel, select it as current list and hit 'Show them'
button, to see those things selected in the list.
You can also 'Export them' as text form to a file.
Want a more complex selection? For instance, all those that are both
kneeling and also being gazed at? Well, you do this step by step:
- First, make up a list of all those kneeling, with the source being 'all items'. Selection is 'Take those for which the
following are true' and attribute 'Posture' equals 'kneeling'. 'DoIt'
produces a list that you rename as e.g. 'kneelers'.
- Second, use this 'kneelers' list as a source with 'all related' and relationship type of 'gazed at' and
with 'bwd' ticked. Selection is 'Take all of them'. 'DoIt' produces the
list you want, which you might rename 'kneelers looked at'.
Values are the contents of attributes, and can be either
- quantitative or
- qualitative.
Quantitative values are numeric, such as 25, 0.75, 1997, while qualitative
values are non-numeric, such as labels like 'Monday' or longer text
strings. At present Annotator has a wide range of value types that have
been brought over from another program (the
Istar knowledge
based system creator) and most will probably be irrelevant to your needs at
first - such things as compass directions, odds, ratios, probabilities,
bayesians, proportions, etc. - though you are free to use them as you wish.
In future it is planned to add more meaningful types.
A particular type of interest in most applications is the
Enumerator or Ordinal value. This allows you to set up sets of
discrete values like the days of the week or types of weapon ('Sword',
'Bow', 'Axe', 'Dagger'). To do this, bring up the
Domains Management Panel, select type 'Enum' or
'Ordinal', give it a name (e.g. 'WeaponType') and hit the 'Add' button.
Then move ove to the right hand list, enter the name for the first one
(e.g. 'Sword') and hit the right hand 'Add'. Repeat for each type. The
difference between Enumerator and Ordinal is that the latter has a numeric
order while the former doesn't. (So, 'Very Strong', 'Strong', 'Weak' would
be an ordinal while 'Sword', 'Axe', 'Bow' would form an enumerator.)
At present, you can only export lists.
Templates are wee knowledge bases that hold a selection of item, attribute
and relationship types suited to certain types of picture. You can make up
your own, either from scratch or from an existing one. They are like
normal knowledge bases but lacking a picture. When you hit the 'New'
button a template will be loaded to go with your picture; they are located
in the Ann:Templates drawer.
Which template is loaded? Normally the one called
Template.std. So you can copy any of the others to it, as you
need, before starting the Annotator. Or, better, hit the Prefs button on
the main control panel and you can select the file you want to act as a
template.
To create your own template from scratch, you will be given a knowledge
base without any user types, and you create attribute, item and
relationship types in that order:
- Hit the 'BrNew' (Brand New) button on the main control panel.
- When asked for a picture, hit 'Cancel'.
- (You will get a warning that you have no picture; hit the 'Create'
button.) Up comes the KB Panel.
- Hit the 'Att Domains' button. Up comes the Domains
Management Panel.
- Create a number of attributes you think you might need, as explained in
the instructions. When done, hit 'Done'.
- Now create the item types you require, by hitting the 'New' button
under the item types list on the KB Panel.
- Now create the relationship types you require, by hitting the 'New'
button under the relationship types list on the KB
Panel.
- Give the KB a suitable name by hitting the 'KB Info' button on the KB Panel.
- Now hit the 'SaveAs' button on the KB Panel and save
the KB in the
Ann:Templates drawer under a suitable name.
There is one of these panels for each item, scene or list, and any number
can be open at a time (though like any Amiga Window, you must send-to-back
or bring-to-front to find the one you want. Each gives details on one
item and allows you to change things of it. It also allows you to move to
related items' panels.
There are five main parts to the panel:
- Top line: Name and Meaning.
- Top third (except top line): Various details, including its visual
appearance. 'Topics' will bring up a panel, allowing you to attach topics
to the item.
- Panel termination buttons: OK, Cancel and Delete. With Delete you get
rid of the item as a whole (and also its graphic) and of all relationships
attached to it.
- Middle: Two lists of related items: left =
those that point to this one, right = those this one points to.
- Bottom: List of attributes of this item. You
can re-order them and add new attributes, or get rid
of ones you don't want.
The 'See' buttons allow you to traverse to related things. Simply select
the thing and hit the See button. Note that you can add attributes to, and
get rid of attributes from, an item indepentently of its item type.
This panel shows details of a relationship, that is a link between two
items or other linked things. The main thing of interest is its type and
the two things at either end of it. You can alter the type by hitting the
wee button to left of type. Also there is a short list gadget showing
attributes of the relationship, if any.
The 'Delete' button will delete the relationship, and also erase its visual
graphic.
This panel shows details of an attribute, allowing you to set its value or
change its name or meaning. You can also change its type.
To
add a new attribute to an item or anything else:
- Go to the Attributes list in the panel.
- Hit the 'New' button; a panel comes up, inviting you to add an
attribute with the following steps:
- Select its
domain type from the list of domains,
- (When you click a domain, its name is placed in the Name box.)
- (If the domain you want is not there, you can
create a New Domain.
- Enter the Name you want (if different); press Return key.
- Enter a Meaning for the attribute if you wish.
- Hit the OK button.
- .. and the new attribute should appear in the list of attributes.
Domains are sets of values that are meaningful to you. They are what
attributes of items (and relationships) are linked to as their 'type'. You
can create new domains anytime you like, e.g. when adding an attribute to
an item or item type. You can also create many of them at one sitting
using the Domains Management Panel.
You can create new domains while in the middle of adding an attribute and
you see there is no suitable domain; merely hit the New Domain button on
the
Add Attribute panel. Or you can hit the Domains
button on the
KB Panel, which allows you to create several domains at
once, say when setting up a KB, and allows fuller entry of details.
In the
former case, when you hit the New Domain button, a panel
comes up showing a list of Value Types, such as Integer, TextString,
Probability. There are three simple steps:
- Select a Value Type.
- (Its name is placed in the Name box.)
- Enter a suitable Name (if alteration needed).
- Enter a Meaning if needed.
And your new domain should now appear in the Domains list so you can
select it.
In the
latter case, when you hit the Att Domains button, you are
presented with a larger panel, with two lists. The left hand list is of
currently available domains. Below it are three boxes which allow you to
create new domains or modify existing ones. To create a new domain, enter
Name (N) and Meaning (M) and choose Type, then hit the Add button. To
update an existing domain, select it from list, make the changes to Name,
Meaning or Type and hit the Update button.
The right hand of the panel holds lists of labels associated with certain
types of domains called Enumerated or Ordinal types,
such as days of week ('Sunday', 'Monday', etc.). You can add new labels by
entering the label in the N box and hitting the Add button.
(To be written properly.)
Many panels show a standard list gadget, which is a ListView gadget
surrounded by several other ones that take action on the list. In some of
the lists some buttons might not be relevant and so are missing. Also,
there is a slight difference between lists of related items and other
lists. The buttons are as follows:
- See: This will bring up a panel of details of the
selected list element - an item details panel, an attribute panel or even a relationship
instance panel.
- R: This applies to lists of related items. It brings
up details of the actual relationship that links to the selected list
element. Otherwise it has the same action as 'See'.
- X: This will 'explain' the selected thing, but in
current version of Annotator it is no different from 'See'.
- Q All: Only relevant to lists of attributes, this
will bring up a panel that presents all the attributes, inviting you to
enter their values. This is the same panel as comes up when you draw the
item.
- 1: This will move the selected element to first in
the list.
- n: This will swap the selected element with the next
in the list.
- New: This will bring up panels to create a new
element in the list.
- Rid: This will get rid of the list element.
There is one of these KB Panels for each Knowledge Base loaded. It
allows you to do things at the whole-KB level such as saving it and also
selecting the item type or relationship type since these are common to all
pictures in the KB.
Written by, and copyright (c) 1997 of,
Andrew Basden.
Email questions or comments
would be welcome, especially if they aid exploration and enriching of
the ideas. Last modified 5 June 1997. Number of visitors to these pages:
. Written on the Amiga
and Protext.