Sound Studies Group





This is the Sound Studies Group at the SUSTech School of Design, Shenzhen. It is led by Dr. Qiushi Xu and Dr. Marcel Zaes Sagesser. Our research focuses on sound in relationship with technology, the environment, and human listening. It includes sound technology, sound studies, Human Computer Interaction (HCI), sound design, philosophy of science and technology, spatial audio, interactive audio, and creative media. The output of this team is published in prestigious international journals, at domestic and international conferences, and at international exhibitions and concert halls.


→  Members
→  Announcements & News
->  Publications

DS366 Composing Experience 体验建构

Spring 2024


Prof. Marcel Zaes SAGESSER
Office hours Marcel: Tuesdays, 13:00 - 15:00 (sign up 24 hours in advance)
TA: WEN Jiayi 文嘉懿

64 hours (32 lecture hours, 32 practice hours)

Class folder (shared, password protected)
Schedule for Theory Presentations

Software Resources
Illustrator Tutorials EN
Illustrator Tutorials CN
After Effects Tutorials

Course Summary

A practice-based studio course in experience design that focuses on planning, analyzing, combining, communicating, and fluidly translating across materials and media the individual parts that make an intangible experience.

Course Description

The course invites students to consider experiences as constructed from many individual parts. Students learn to analyze and construct time-based and often intangible experiences across various materials and media. By combining and recombining single components of experiences, students practice translating ideas fluidly between means, media, formats, and technologies to support the planning, realizing, analyzing, and communication of their ideas. A variety of visual and language means is used to render visible that which normally is invisible or intangible. Formats from different fields, such as plans, maps, models, diagrams, storyboards, musical scores, animated plans and others are studied in detail. At the end of this course, students are able to compose and communicate experience design at an advanced level. They will demonstrate competence in moving from planning to implementing, communicating, and back; and they will develop fluency in multimodal representation as well as critical thinking.

Learning Outcomes

  1. Name and explain a variety of tools that are available to experience designers for communicating their ideas
  2. Demonstrate experimentation in visual expression as compositional device
  3. Demonstrate critical thinking and precise communication skills 
  4. Develop, realize, and present a composed experience that successfully communicates your idea to a potential audience



Weekly Plan


1

Course Introduction: meaning of “composing”
Exercise in composing

  1. Welcome, expectations
  2. Definitions of “experience” and “composing” - Exercise in composing
  3. Logistics: overview of the course content, practice project, and the expectations about academic, ethical and creative rigor   
  4. Assignment of Student Theory Presentations


Pencil sketches by Emily, Judy, Leo


2

Components of an experience
Splitting a whole in its individual parts

  1. Theory Presentations (examples, case studies, classroom discussion)
  2. Project work: sketching with pencil on paper
  3. Project work: sketching with pencil on paper
  4. Project work + Logistics + Presentation

Homework:
  • Read pdf summaries
  • Bring project to class



Illustrations/Diagrams by: Zoe, Renchao, Anna, Lance, Judy, Mia, Wag, Emily, George, Vincent


3

Sketching across materials
Material exploration

  1. Theory Presentations (examples, case studies, classroom discussion)
  2. Project work: 2d, illustration
  3. Project work: 2d, illustration
  4. Presentation and discussion of results

Homework for this week:
  • Read pdf summaries
  • Bring project to class (and decide which project)
  • Bring 1 text paragraph to class, describing your project to a stranger (in English)



    Samples shown here by Marcel (mind map on the left), and by George, Judy, Kane, Leo, Rondo, Renchao, and Anna


    Exercise:
    Select 1 material of your project that is crucial for the experience that you are composing
    Find at least 2 different ways to represent this material on the flat paper as meaningful as possible

    Requirements:
    • A3 landscape
    • Split in 3
    • Layout: 
      Left column: author name, title, subtitle, design field, short description
      Middle+right column: free space to communicate experience
    • Need be computer drawing, not pencil
    • Need to look professional
    • Need to be in a presentable shape this afternoon.



    4

    Sketching across media
    Media exploration

    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Study of media examples, and demos in AE (Adobe After Effects)
    3. Project work: time-based, explore media in time
    4. Presentations and discussion of results



    Work shown is inspired by Hannah’s project


    Case Studies

    Audio Branding, interactive web presence
    Modular Theatre (by Orey)

    Task: Introduce time to your illustration, bring together different media + formats in a single composition (short animation)
    Keywords: Vectorization, Tracing, Picture-to-Vector, Animation, Drawing, Transition, Time

    Homework:
    • Revise your sketch from this week’s class. React to the feedback. Upload the new version to the folder “exercise / week 4 revision”
    • Each of you, describe 1 tool/function inside Illustrator, and 1 tool/function inside Figma, that you have used or that you have just learned about, and that you think is special and helps others in their projects, too. Put them with a screenshot and a short text description (EN+CN) on the class note document under Week 4: https://docs.qq.com/doc/DSmlpTk1FZE5YWUZO




    5

    Translating ideas
    Develop fluency

    Homework
    • Watch 2 AE After Effects Tutorials (from the official Adobe website) and add 1 screenshot and 1 short description to the Class Notes Document (Instead of 2 AE tutorials, you can also watch 1 AE and 1 Figma-to-AE tutorial).
    • Create a draft animation of your project in which at least 2 elements are moving in order to communicate something
    • Read 2 theory pdfs by Anna & Renchao


    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Developing Fluidity in Sketching across materials, static and time-based
    3. Project Work (with After Effects Tutorial)
    4. Project Work (with After Effects Tutorial)

    Topics:
    • Paper Scribble
    • Sketching (as leaving out, abstracting, simplifying, reducing)
    • Adding scribble to animation
    • Adding text to animation
    • Adding photos to animation
    • Animate single vector point in path drawing (AE)


    Work samples shown here by Zoe, George (paper scribble), and by Lance, Connor, Judy, Emily, and Zoe (animation drafts)


    6

    Overview of tools for experience designers
    Naming, understanding, and using tools

    Homework:
    • read theory pdfs before class
    • watch 2 additional AE tutorials that you have not watched yet, and add key elements to the class document (you can also add this document to find inspiration from other class members)
    • individual project: revise animation, make big progress. Make it longer, better, more expansive. Add at least 3 new elements from this list: text/annotations/keywords, arrows, lines, background color boxes, backgrounds, photos, photorealistic textures, animated textures, etc.). Respond to the feedback you have received. Goal: your animation should communicate 1 aspect of your project really well, without you having to explain it (the animation should be able to stand on its own and communicate to someone who doesn’t know your project at all). When you use that, use the scribble excercise from yesterday's class as a tool to decide which elements can be left out, which ones can be abstracted, and which ones need be shown in fullest detail?

    Today:
    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Workbook: Tools for Designers
    3. Workbook: Tools for Designers
    4. Animation review (homework review) & next steps; info about next week’s presentation

    Watched + discussed Adobe After Effects tutorials: 
    what is possible with AE, Bring your brand to life, Draw complex shapes, Keyframe animateion, Add realism with motion blur



    Image compositions shown here made by Marcel (water melon), Zoe, Rondo, Leo, Renchao, Kane, Emily


    7

    Student presentations, with feedback by several (external) reviewers:

    Exhibition-style showcase of your project: you are not allowed to talk, rather you are required to show a video that can speak for itself. Video duration requirement: between 30 seconds and 2 minutes. Resolution: HD 1080p (16:9). With or without sound. The video should include, and combine, many elements of the following: title of project, student name, project field (example: product design, UX design), text, text annotations, keywords, drawing, animation, photo elements, etc. Use and apply all the skills learned in this class.

    This is a first draft of your “Large Project” for this course:

    Prompt: Each individual plans, realizes, presents and submits a large-scale design project: a larger ‘plan’ of an experience chosen by the individual, which involves different digital media, formats, forms, and styles to communicate the experience idea to a public that doesn’t know about the experience. The most common deliverables will be a video presenting, or being, the plan, along with a written report. Alternative deliverables are accepted if they are discussed ahead of time with the instructor (potentially VR projects, apps, series of drawings, and many others are accepted if there is a strong reason why they cannot be presented within a video).
    The progress of this large project will be presented to the class in an interim presentation (week 7) and a final presentation.


    Excerpts from Lance, Anna, Hanna, and Mia



    8

    Learning from feedback:
    Presenting ideas

    Homework for today: prepare systematic responses and priorities to feedback received last week at the presentation. Come to class prepared with a list and prepared to discuss your list. 

    Today:
    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Case studies (Bose + Samsung), feedback on technicality that should be shown, input: Chapter 6.5, “The Review” (presenting ideas) – elevator pitch. Exercise: write someone else’s elevator pitch.
    3. Project work & tutorial meetings
    4. Project work & tutorial meetings



    Homework for next week:
    • Revise your video / extend it accoring to the feedback as discussed today. For many of you it involves shooting video, creating new animation parts, adding technical details, and illustrations. For some it requires to add sound. Upload the new version before the next class start to the folder "week 9 revisions"
    • Write a very short, but good, elevator pitch. It has four parts, according to our theory book. Upload your version before the next class start to the folder "week 9 elevator pitch".
    • Read the two pdf theory summaries that will be circulated next week before class.



    9

    Analyzing experiences
    In-depth analysis of case studies

    Today:
    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Case studies: analyses of deep research in design that involves plans, sketches, illustrations, annotations, animation
      Discuss homework, elevator pitch, revisions
    3. Project work & tutorial meetings
    4. Project work & tutorial meetings



    10

    Planning, combining, recombining, remixing
    Moving back and forth between plan and “thing”

    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Project work & Tutorial meetings (discuss remaining elevator pitches)
    3. Project work & Tutorial meetings
    4. Project work & Tutorial meetings



    11 (!)

    Watch out: no class on May 1st, but class is postponed to Saturday May 11th
    Communication skill: imagine your audience
    Present your work


    12 (!)

    Class on Wednesday + Saturday this week
    Critical thinking
    Text writing


    Special: Music/Sound Composition Workshop this week


    13

    More tools: survey of advanced tools for experience designers

    14

    Coursework submission, with classroom discussion

    15

    Composing experiences: moving between making and analyzing

    16

    Work documentation & submission


    17/18 - Exam Weeks

    Final exam, date and time to be announced.


    Core Readings


    • W. Buxton, Sketching User Experiences: getting the design right and the right design. Amsterdam Boston: Elsevier/Morgan Kaufmann, 2007.    
    • S. Greenberg, S. Carpendale, N. Marquardt, and B. Buxton, Sketching User Experiences. The Workbook. Elsevier, 2012. doi: 10.1016/B978-0-12-381959-8.50001-8.  

    Readings: Choice for Student Theory Presentations


    • Berridge, G. Events Design and Experience. (Butterworth-Heinemann, Oxford ; Burlington, MA, 2007).        
    • Bowles, C. & Box, J. Undercover User Experience: Learn How to Do Great UX Work with Tiny Budgets, No Time, and Limited Support. (New Riders, Berkeley, CA, 2011).        
    • Ciuccarelli, P., Lupi, G. & Simeone, L. Visualizing the Data City: Social Media as a Source of Knowledge for Urban Planning and Management. (Springer International Publishing, Cham, 2014). doi:10.1007/978-3-319-02195-9.    
    • Chapman, J. Emotionally Durable Design: Objects, Experiences and Empathy. (Routledge, 2015). doi:10.4324/9781315738802.
    • Conan, M. Landscape Design and the Experience of Motion. (Dumbarton Oaks Research Library and Collection, 2003).        
    • Participatory Design Theory: Using Technology and Social Media to Foster Civic Engagement. (Routledge, London New York, 2019).        
    • Mapping the Unmappable?: Cartographic Explorations with Indigenous Peoples in Africa. (transcript publishing, Bielefeld, 2021).        
    • Dohr, J. H. (Author). Design Thinking for Interiors : Inquiry, Experience, Impact.        
    • Dunne, A. Hertzian Tales: Electronic Products, Aesthetic Experience, and Critical Design. (MIT Press, Cambridge, Mass, 2005). 
    • Impett, Jonathan: Sound Work.Composition as Critical Technical Practice. artisticresearchreports.blogspot.com (2022).        
    • Jason Farman. Information Cartography: Visualizations of Internet Spatiality and Information Flows. in Composing(media) = composing(embodiment): bodies, technologies, writing, the teaching of writing (eds. Arola, K. L. & Wysocki, A. F.) (Utah State University Press, Logan, Utah, 2012).        
    • King, D. L. Designing the Digital Experience: How to Use Experience Design Tools and Techniques to Build Websites Customers Love. (CyberAge Books/Information Today, Medford, N.J, 2008).   
    • Koolhaas, R. Delirious New York. (Monacelli Press, New York, New York, 1994).     
    • Llano, S. Mapping street sounds in the nineteenth-century city: a listener’s guide to social engineering. 20 (2018).        
    • Lombardi, V. Why We Fail: Learning from Experience Design Failures. (Rosenfeld Media, Brooklyn, N.Y, 2013).        
    • Design, User Experience, and Usability. Theory, Methods, Tools and Practice: First International Conference, DUXU 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part II. vol. 6770 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2011).        
    • McMurray, P. Ephemeral cartography: on mapping sound. SOUND STUDIES 34 (2018).        
    • Ouzounian, G. Mapping the Acoustic City: Noise Mapping and Sound Mapping. in Stereophonica: sound and space in science, technology, and the arts (The MIT Press, Cambridge, Massachusetts, 2020).        
    • Robinson, S., Marsden, G. & Jones, M. There’s Not an App for That: Mobile User Experience Design for Life. (Elsevier/Morgan Kaufmann, Amsterdam ; Boston, 2015).        
    • Simon, M. Storyboards Motion in Art.        
    • Tabački, N. Consuming Scenography: The Shopping Mall as a Theatrical Experience. (Methuen Drama, London New York, 2020).        
    • Ward, P. Picture Composition for Film and Television. (Focal Press, Oxford ; Boston, 2003).        
    • User Experience Re-Mastered: Your Guide to Getting the Right Design. (Morgan Kaufmann Publishers, Burlington, MA, 2010).        
    • Xenakis, I. et al. From Xenakis’s UPIC to Graphic Notation Today. (Hatje Cantz, Berlin, (Germany), 2020).        
    • Yu, J. Shopping Experience: Store & Showroom. (Design Media Publishing, 2011).  


    Assessment

    • Attendance 10%
    • Final Exam 20%
    • Coursework 70%

    Attendance

    Attendance: 10%
    • 100 points for never late and never missing a class
    • 100 points for being up to 4 times late (up to 50min), OR for missing 1 entire class
    • 50 points for missing 1 entire class and being up to 4 times a little late (up to 50min)
    • 50 points for never missing a class, and being up to 8 times a little late (up to 50min)
    • 0 points for missing 2 or more classes
    • 0 points for being late for more than 8 times
    *all late or missing when unexcused. If excused with a reason, it doesn’t count toward the attendance grade.

    Large Project

    Prompt: Each individual plans, realizes, presents and submits a large-scale design project: a larger ‘plan’ of an experience chosen by the individual, which involves different digital media, formats, forms, and styles to communicate the experience idea to a public that doesn’t know about the experience. The most common deliverables will be a video presenting, or being, the plan, along with a written report. Alternative deliverables are accepted if they are discussed ahead of time with the instructor (potentially VR projects, apps, series of drawings, and many others are accepted if there is a strong reason why they cannot be presented within a video).
    The progress of this large project will be presented to the class in an interim presentation (week 7) and a final presentation.
    [detailed prompts with grading criteria follows later]

    Student Theory Presentations

    Each student sings up for a presentation of a theory topic. Each topic is based on an academic book or article on different forms of plans, experiences, time-based media, and experience composing. The prompt is for each individual in the class to choose a book/article, understand it thoroughly, and summarize the key take-aways in a short presentation for the peers. If it’s a book with many chapters or different essays, it’s better to choose one of them. The focus should be on how do we plan a time-based experience with visual presentation? How do plans and sketches look like? Is is the plan used in communicating the experience? And how does the plan/sketch feed back into the experience? Presentations are max 10 minutes, with a following 10 minutes for in-class discussion, moderated by the student.

    1. from the reading, choose the most important text parts and images (roughly 3 pages of text from the book, and 5 images from the book)
    2. create a handout for your peers with the choosen text and images (name sources and page numbers from original, and don’t change the text). If you add your own text to it, make sure you label clearly which is your own, and which is from the book.
    3. create a Powerpoint (or pdf) slide show that you will use to present the key ideas from the reading to the class
    4. present for max 10 minutes.
    5. after that, lead a class discussion (another 10 minutes). The best is to prepare 3-4 good questions that help you be a moderator.

    Submissions:

    • Until Friday evening 18:00: briefly discuss your choice of text with the instructor (email)
    • Upload handout for class: until Sunday night 23:59, to the class folder
    • Upload slides: until 14:00 on class day (Wednesday)


    Final Exam

    One part of the written exam is based on all the readings (academic books and articles) that have been presented in the student theory presentations. The second part is based on the core reading of the class, presented by the instructor. The third part of the written exam is software knowledge of the tools taught and used in this class (primarily Adobe After Effects).
    [Details will be added later]


    Classroom Policy

    After each class, the classroom has to be cleaned up properly. Everyone is responsible for their computer and their desk, chair, and the floor around them. Remove any food waste, any trash. Make order on the computer, on the desk. If the keyboard/mouse are low on battery, put them on charge. Backup your files privately, put the chair nicely to the desk.
    No eating during class time. Drinking is always allowed. The breaks can be used for eating.

    Equipment and Device Policy

    Technical devices are here to be used by the members of this class. They can be used freely during class time, and outside of class time, they can be either accessed in the classroom, or some of them can be checked out and can be taken with you. TA Jiayi is responsible for the device checkout.

    Academic Integrity

    https://msagesser.github.io/ds226-2023/SD-late-policy.pdf
    Sound Studies Group
    SUSTech School of Design
    All content © 2023 Sound Studies Group