Cognitive Foundations of Visualisation (TU Vienna, Austria)
99 flashcards · Computer Science · by Locke Lamora
History of using pictures
Pictures controversial: some religions forbid them some religions used them broadly Catholics had bible of the poor (text + pictures) Old books had problems with integration of pictures (printing press was for text, woodprint for pictures) Before central perspective: multiple angles at same time (perpetuum mobile) Central perspective in renaissance After central perspective: show temporal development (lady stairs) Pictures are used to identify when things were invented
Scientific illustrations
- Observation: simplest: drawing - Induction: images help answer why and how questions - Methodology: Show systematic approaches (how circulation of blood works) - Selfillustration: Trees have rings in wood, a phenomena that illustrates itself - Classification: Put items into classes to be able to compare (picture of evolution) - Development of concepts: concepts allow us to reflect on observations and develop theories, allows conveying theories
Difference verbal and pictorial representation
Pictures also need to be learned! (boy watering plants)
Explain analogous pictures and logical pictures
- analogous pictures: picture resamble depicted facts - logical picture: shows concept that can not be perceived otherwise (bar chart,...)
Visual literacy
- literacy: ability to read and write - Visual literacy: ability to understand and interpret images - Digital literacy: ability to understand and use information in multiple formats from multiple sources - History: Kodak company supported research how children understand pictures, supported visual literacy - Visual literacy has multiple areas: Visual thinking, visual learning (ignore), visual communication
Four different aspects of visual literacy
visual literacy as ability to understand visual media enhancement of cognitive abilities through visual literacy A better visual literacy is beneficial for cognitive development Raising awareness of manipulation through increased visual literacy identify manipulation Aesthetic appreciation It helps enjoy art and other visual material
Visual Learning
how to design appropriate visual learning materials and how learners decode such materials.
Visual Thinking
Different opinions: Perception comes first, then based on the perception thinking process start - clearly distinguishable In early stages of perception through processes occur. For example active selection, we decide what we perceive (focus on vehicles when crossing street, not colour of road). Other example is concept formation (object needs to have organised shape to be perceived)
Visual Communication
how messages are communicated by visual material: designs often work subconciously examples: objects or persons seen from an oblique angle are seen as more detached and distant than objects or persons seen from the front The person in the picture is more tightly related to you and tries to reach out to you The picture conveys the erroneous impression that Lise Meitner was the assistant of Otto Hahn.
Properties of direct sensory perception:
- understanding without learning (hard-wired in brain) - behaviour can not be unlearned (optical illusions) - sensual immediacy (perceived very quickly, orientation and shapes helps to attract attention and point out differences) - Studied through neuroscience
Properties of conventional representation:
- hard to learn (writing) - easy to forget (formulas, some not like reading direction) - embedded in cultural context (arabic numerals, reading directions, hard to replace) - powerful form of representation (mathematical representation) - rapid changeability (changes over course of time) - Investigated using psychology and sociology
What is low level vision
addresses more simple form of vision like perception of colour related to unconscious and automatic types of vision acquired in early stages of development question: is perception innate or learned
The eye
- Retina (light sensitive layer, rods [light] and cones [colour]), image is reversed and inverted - Pupil, Lense (controls light and reflection) - Rods and Cones (more rods than cones) - Fovea centralis (highest acuity of vision, only cones, point of focus)
Visual Field
- some seen with both eyes, parts only with one eye - both eyes needed for depth seeing (only works nearby not far away) - some depth information we get from colour, texture,... but those are learned from the brain
Rules of perception
a straight line in picture should be interpreted as straight line in 3D if the ends of two lines coincide in a picture, then interpret them as if the coincide in 3D
Lateral Inhibition:
- Retina emphasises contour lines by increasing contrasts - Helps emphasise tiger - Ganglion cells inhibit their neighbours (not only upwards)
Colour Perception
- Spectral colours - Monochromatic colours, light of only one wavelength - polychromatic colours, mix of multiple wavelengths - Colours in middle of spectrum (green, yellow, orange) are brighter than colours on edge - Needed to see apples much more easily :P
Properties of colour perception
- shade: Depends on wavelength - saturation: expression for number of different wavelengths, lower the more - brightness: light intensity
What colour theories are there
Two theories, they do not contradict each other, both apply Trichromatric theory Opponent process theory of colour vision
Trichromatric theory
Assumes each hue can be mixed by light of three different wavelengths (RGB) Inverse true for mixing paint Need of three different cones in eye, for different colours
Opponent process theory of colour vision
Explains simultaneous and successive contrasts, afterimages and red-green blindness Assumes that certain mechanisms produce opposite responses due to opposing neurons, complex process, no need to know in detail neurons are in corpus geniculatum laterale (back of head)
Colour Constancy
- colours do not change much depending on lighting, brightness,... - due to colour pigments of cones fade when exposed to light of certain wavelength for longer period of time - if long in yellow light, things won't look as yellow any more
High level vision
Distinction between conscious – unconscious perception perception is an active process: (development together with musculosceletal system, eyes move frequently - saccades) orientation: constant movements with head and eyes, ability to predict events, perception supports goal oriented and exploratory movement
Strengths and weaknesses of human perception
- perception developed during evolution, therefore focused on movement - consistens information is less well perceived - good at recognising visual patterns and distinguishing between different ones - can not recognise radio waves, rays, magnetic forces, ultraviolet rays,...
General information about of object perception:
- same object from different angles (object consistency), not trivial - Higher level than pattern recognition or colour perception - only focusses on parts of field of view - very difficult to learn (people who were blind for first few decades of life) - Single out one object in view - Uses shape (most important), colour, texture and location of objects - highly based on experience, expectation and context - relationship is also important (what type of cat,..)
Different theories of object perception
"simple theory" Feature Integration Theory Recognition-by-Components Approach Alignment Method Assumptions for 2-4 is always that you identify some kinds of regularities Most likely 2-4 are (partly) true
"simple theory" of object perception
We compare it to previous "photos" of object This is not true because: number of possible views of an object is very large views are not similar enough to recognise object changing shape (movement of object/person)
Feature Integration Theory
looks at 2d features (curvature, end of line, orientation,...) compares perceived object with information in memory
Recognition-by-Components Approach
looks at 3d geons (cylinders, spheres,...) we are able to recognise objects by separating them into geons geons are view invariant are discriminable (geons are distinguished from each other) resistance to visual noise (even recognisable in poor visibility conditions) criticism: only allows to distinguish between classes, not between similar objects
Alignment Method
only one canonical image of object O is in mind, model M object model M only allows certain transformations (size, position, orientation) example of character recognition (A in different sizes, scale, transformations,..)
Gestalt psychology
Basic Idea: The whole is more than the sum of its parts perceptual system processes visual information from the environment, it adds structure to these stimuli the stimuli that affect a human must always be seen as a whole several gestalt laws
Gestalt law
Law of figure and ground Figure is more thing-like than ground Figure is in front of ground Ground is seen as unshaped material Contour of figure is part of figure Law of good continuation We see lines not angles Law of proximity Elements are perceived as belonging together when they are spatially close to each other Law of similarity Elements similar to each other are perceived as belonging together Law of good form (gute Gestalt) Simplest geomatrical forms are perceived, not complex ones
Perceptual cycle
Model of Broadbent
Data driven vs. conceptually controlled processing
Ecological approach to perception (Gibson)
Experiments have to be done in natural environment Argues that perception and movement are related movement is important due to different angles (invariance, we always perceive same object), different textures and movement creates optical flow field Concepts: ambient optic array texture affordance invariance
Ecological Approach: Ambient Array
only see part of the object, when moving experience different parts of object
Ecological Approach: affordance
function can be derived directly from shape of object, not constructed by the brain, but in the object
Ecological Approach: invariance
objects remain constant under transformations (lightness, size, time-movement, shape, color constancy) structural invariance (objects reflect light in different way) transformational invariance (certain moves let you identify objects/humans) shape constancy (observer moves, but object does not)
Models of Information processing approach
1. Model of Broadbent 2. Model of Selfridge 3. „Pandemonium system“ (Lindsay & Norman) 4. Data driven vs. conceptually controlled processing 5. Perception cycle of Neisser
Model of Selfridge
Similar to pandemonium (will be ignored) Assumes most commonly used patterns are composed of few elements (vertical bars, right corners, left corners...) These retain meaning when size is changed and other transformations Brain has developed detectors for these, that work in parallel Example reading text: first read character, then understand word, then put into context,.. Context, grammar and prediction helps speed up process
Ecological Approach: Texture
is usually regular and helps determine distance
constructivist theories of perception
rather top- down approach perception is active and constructive process not directly determine by stimuli affecting sensory organs ---> product of interaction between stimuli, internal hypotheses, expectations/knowledge,... perception can be flawed contextual information is important
Direct perception (Gibson)
- bottom up theory - light striking the eye - optical flow field - OFF contains unambiguous or invariant information - Information is received in direct way (no process of information processing) - meaning arises through prompting character of object not through memory - Could not explain optical illusions - can not explain meaning of objects (sofa with kids pretending it is a spaceship)
Dual Coding Theory
Assumes that two memories: one for verbal material one for images "How many windows are there in the apartment you live in"
Memory for pictures
Easier to remember pictures than texts example study with US yearbooks, pictures were easier to remember than text possible to supplement visual memory with verbal description
Imagery Debate
One side argues: everything is translated into verbal form in brain Assumes mental images work like photographs (nobody argues that) translated into propositions other side: mental images in brain information stored like abstraction of image propositions are not able to store visual information well example: Mental rotation, figure out whether two objects are the same, rotate them in brain
construcivist theories: discussion
problem: perception usually correct --> not plausible in the context high value in explaining perceptual illusions, many predictions empirically proven valuable when stimuli only presented briefly or are poorly recognizable
Differences Pre-attentive & Attentive Processes
Pre-attentive Processes: are executed automatically; without the influence of consciousness; very short-term; immediate image interpretation; Attentive Processes controlled search; explicit processing, greater mental effort;
pre-attentive processes
parallel processes of perceptual organization, pattern recognition and object identification global meanings processed before details schemata reduce possiblities of interpreting pictoral signs not just "recording process", but also interacts with higher processes like knowledge and hypotheses mechanisms of pre attentive processes that control attention: color orientation size shape direction sharpness
Attentive Processes
Observer tries to extract task-relevant information from pictures Individually select, analyse and interpret image elements Image processing can be improved by instructions
Perception and social influences
There are indications that perception is influenced by social and emotional factors (Poorer kids perceive coins larger than richer kids)
Combination of the senses
All senses influence perception Information visualisation only uses one of them Assumption that visual sense is most important is simplistic
combination of image and text
images should be explained in detail and not contradict text methods to combine images and text_ redundancy complementarity supplementary juxtraposition stage setting
Theoretical positions on combination of senses
doctrine of equivalent information Assumes that different senses inform about same characteristic with same value doctrine of analogous qualities Senses have some nonspecific information and are not identical to other senses doctrine of corresponding psychophysical properties perceptual intensity increases not linearly but logarithmically by adding senses corresponding information Each subsystem adds information (fire is smelled, seen, heard,...)
Redundancy
content is basically the same essential ideas repeated beneficial for complex content annoying for simple content
Complementarity
text and image have different contents contents complement each other neither text nor picture self explanatory --> only work together
Primacy of Visual Perception
Visually can not process more data than auditory because information is not identical, so can not be compared Hearing is a time-dependent process, seeing not, can not be compared Hearing guides visual system (look towards sound) Auditory System and visual system support each other
stage setting
involves using visual elements to set a specific context or atmosphere for a topic used to activate relevant prior knowledge and create a context for new information advanced organizers used to activate context at beginning of chapter/lesson example for a manual for troubleshooting VCRs
Supplementarity
either text or image plays a more important role, other one has supporting effect used when emphasizing certain points helpful for learning
Juxtraposition
placing two or more contradictory elements close to each other to attract attention and create a contrasting effect frequently employed in advertising also useful in educational settings
Benefits of Visualisations
1. Larger cognitive resources 2. Faster search processes 3. Improved pattern recognition 4. Perceptual inference 5. Perceptual monitoring 6. Interactivity
Benefits of Visualisations Larger cognitive resources
increases short-term memory large amounts of data can be more easily perceived
Benefits of Visualisations Faster search processes
Information is grouped Easier to find specific values
Benefits of Visualisations Improved pattern recognition
Organised data -> easier pattern recognition recognise instead of remember
Benefits of Visualisations Perceptual inference
Draw inference directly from visualisation (even in parallel). See in above timeline, what happened when
Benefits of Visualisations Perceptual monitoring
Large amounts of data can be easily surveyed if visualisation is good
Benefits of Visualisations Interactivity
Most of current visualisation are interactive selection of data to be represented selection of methods of representation selection of level of detail
change blindness
humans often unable to detect changes in environment many empirical experiments (blank screen between images,...)
inattentional blindness
attention focused on certain task --> other objects not noticed eg. when driving a car or assessing statements of witnesses
Design Guidelines
items should be manageable and easy to identify visual events should be minimised attention is limited, so highlight right object at right time
Quantitative Evaluation Methodologies
Experiment Questionnaire Logfile-Analysis Eye-Tracking
Qualitative Evaluation Methodologies
Interviews Focus groups Observation Thinking Aloud
How to perform evaluation
Define research question Choose evaluation method Either qualitative (gain more detailed information) or quantitative (easy to generalise)
Questionnaires
Use specific questions, not general ones. Avoid acquiescence bias (tendency to agree). Limited for assessing software quality.
Logfiles
Record keystrokes of users. Interpret behaviour carefully. Watch for unintentional interactions.
Eye-Tracking
Shows where users look. Analyze scanpaths but challenging. Unclear if interest or confusion.
Thinking Aloud
Participants talk while using visualization. Provides insights into users' intentions. Intrusive, requires training.
Interviews
Clear research goals and questions. Develop interview questions. Useful for detailed feedback.
Focus Groups
Group interviews. Ensure participation from all. Similar to interviews.
Observation
Video recordings and coding. Interpret interactions. Use screencapture for support.
Novel Evaluation Methods
Beyond time and error (how many errors are made and how much time it takes) Explore data thoroughly. Ill-structured domains need alternative evaluation methods.
definition: information visualisation
based on abstract data, no analogy to the data presented
scientific visualisation
primarily based on physical data, such as the human body, the Earth, molecules, weather phenomena, and more. aims to represent spatial phenomena or geometric data.
Visual analytics
came from fighting terrorists find patterns in lots of data combining computer program with people (computer preselects cases,...) Challenges: exploratory approach open solution space problem solving as generation of insights
Visual analytics definition
Visual Analytics is the science of analytical reasoning facilitated by interactive visual interfaces. Short: information visualization + analytical component
History of Visual Analytics
Dr. John Snow and Cholera
Interaction methods:
scrolling, panning, zooming filtering data reordering overview & detail (see map of whole country and click on state to have map of state) focus and context (hyperbolic tree) dynamic queries multiple views (with brushing and linking, select in one view, gets selected in other view as well)
Types of InfoVis
geometric techniques (parallel coordinates) icon techniques (Chernoff faces, stardinates) pixel-oriented techniques (density of colour shows attributes) networks/graphs (also matrix representation) hierarchical data (treemap) dynamic representation (animations) 3D representations
mapping in information visualisation
transforming raw data into data tables and finding an appropriate visual analogy, known as a visual structure
Data refinement in mapping
Data refinement is achieved through view transformation, which includes actions like positioning, scaling, and incorporating interactive elements into the visualization.
data types considered in mapping
Nominal Ordinal quantitative
Problems of InfoVis
Incomplete data Corrupt data Data cleansing is very time-consuming Data in space and time
mapping quality primary aspects how to avoid unwanted interpretations
data expressiveness (preserving data through visual structure) human understanding (effective mapping) to avoid unwanted interpretations: be careful when designing a mapping /visualisation
Challenges in Representing temporal data
Occlusion with overlapping routes Visibility of street labels Theoretical framework: Gibson's theory of perception Importance of real-life analogs in interactive visualization
Gibson's Theory of Perception in Interactive Visualization
Connection between perception and movement User preferences for real-life analogs in interaction Actions like zooming in, zooming out, panning Avoidance of interactions without real-life analogs
Route and Survey Perspective in Spatial Perception
Route perspective (e.g., navigation system use) Survey perspective (overhead map view) Prioritizing global features in perception
Benefits and Limitations of Hierarchical Models
Cognitive load reduction through hierarchy Occasional errors due to hierarchy Example: Reno and San Diego location assumption Overall advantages of hierarchical maps
Traditional Maps vs. Specialized Visualizations
Inadequacy of traditional maps for combined spatial and temporal data Specialized visualizations like matrices and augmented matrices Improved representation of data on people meeting at locations over time Comparing traditional maps to specialized visualizations