Sound Studies Group


→ Activities
-> Publications
→ Members
→ Gallery
→ Contact

This is the Sound Studies Group at the SUSTech School of Design, Shenzhen. It is led by Dr. Qiushi Xu and Dr. Marcel Zaes Sagesser. Our research focuses on sound in relationship with technology, the environment, and human listening. It includes sound technology, sound studies, Human Computer Interaction (HCI), sound design, philosophy of science and technology, spatial audio, interactive audio, and creative media. The output of this team is published in prestigious international journals, at domestic and international conferences, and at international exhibitions and concert halls.

#Sound Studies
#Creative Media
#Digital Media Arts
#VR + Sonic Architectures
#Interaction Design
#Sound Design
#Auditory Interaction
#Human Computer Interaction HCI


We are continuously looking for exceptional students for short-term or long-term internships and visiting student positions in our group. Please send us your CV and your portfolio if you are interested.

DS366 Composing Experience 体验建构

Spring 2024


Prof. Marcel Zaes SAGESSER
Office hours Marcel: Tuesdays, 13:00 - 15:00 (sign up 24 hours in advance)
TA: WEN Jiayi 文嘉懿

64 hours (32 lecture hours, 32 practice hours)

Class folder (shared, password protected)
Schedule for Theory Presentations

Software Resources
Illustrator Tutorials EN
Illustrator Tutorials CN
After Effects Tutorials

Course Summary

A practice-based studio course in experience design that focuses on planning, analyzing, combining, communicating, and fluidly translating across materials and media the individual parts that make an intangible experience.

Course Description

The course invites students to consider experiences as constructed from many individual parts. Students learn to analyze and construct time-based and often intangible experiences across various materials and media. By combining and recombining single components of experiences, students practice translating ideas fluidly between means, media, formats, and technologies to support the planning, realizing, analyzing, and communication of their ideas. A variety of visual and language means is used to render visible that which normally is invisible or intangible. Formats from different fields, such as plans, maps, models, diagrams, storyboards, musical scores, animated plans and others are studied in detail. At the end of this course, students are able to compose and communicate experience design at an advanced level. They will demonstrate competence in moving from planning to implementing, communicating, and back; and they will develop fluency in multimodal representation as well as critical thinking.

Learning Outcomes

  1. Name and explain a variety of tools that are available to experience designers for communicating their ideas
  2. Demonstrate experimentation in visual expression as compositional device
  3. Demonstrate critical thinking and precise communication skills 
  4. Develop, realize, and present a composed experience that successfully communicates your idea to a potential audience



Weekly Plan


Week 1

Course Introduction: meaning of “composing”
Exercise in composing

  1. Welcome, expectations
  2. Definitions of “experience” and “composing” - Exercise in composing
  3. Logistics: overview of the course content, practice project, and the expectations about academic, ethical and creative rigor   
  4. Assignment of Student Theory Presentations


Pencil sketches by Emily, Judy, Leo


Week 2

Components of an experience
Splitting a whole in its individual parts

  1. Theory Presentations (examples, case studies, classroom discussion)
  2. Project work: sketching with pencil on paper
  3. Project work: sketching with pencil on paper
  4. Project work + Logistics + Presentation

Homework:
  • Read pdf summaries
  • Bring project to class



Illustrations/Diagrams by: Zoe, Renchao, Anna, Lance, Judy, Mia, Wag, Emily, George, Vincent


Week 3

Sketching across materials
Material exploration

  1. Theory Presentations (examples, case studies, classroom discussion)
  2. Project work: 2d, illustration
  3. Project work: 2d, illustration
  4. Presentation and discussion of results

Homework for this week:
  • Read pdf summaries
  • Bring project to class (and decide which project)
  • Bring 1 text paragraph to class, describing your project to a stranger (in English)



    Samples shown here by Marcel (mind map on the left), and by George, Judy, Kane, Leo, Rondo, Renchao, and Anna


    Exercise:
    Select 1 material of your project that is crucial for the experience that you are composing
    Find at least 2 different ways to represent this material on the flat paper as meaningful as possible

    Requirements:
    • A3 landscape
    • Split in 3
    • Layout: 
      Left column: author name, title, subtitle, design field, short description
      Middle+right column: free space to communicate experience
    • Need be computer drawing, not pencil
    • Need to look professional
    • Need to be in a presentable shape this afternoon.



    Week 4

    Sketching across media
    Media exploration

    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Study of media examples, and demos in AE (Adobe After Effects)
    3. Project work: time-based, explore media in time
    4. Presentations and discussion of results



    Work shown is inspired by Hannah’s project
    Case Studies
    Audio Branding, interactive web presence
    Modular Theatre (by Orey)

    Task: Introduce time to your illustration, bring together different media + formats in a single composition (short animation)
    Keywords: Vectorization, Tracing, Picture-to-Vector, Animation, Drawing, Transition, Time

    Homework:
    • Revise your sketch from this week’s class. React to the feedback. Upload the new version to the folder “exercise / week 4 revision”
    • Each of you, describe 1 tool/function inside Illustrator, and 1 tool/function inside Figma, that you have used or that you have just learned about, and that you think is special and helps others in their projects, too. Put them with a screenshot and a short text description (EN+CN) on the class note document under Week 4: https://docs.qq.com/doc/DSmlpTk1FZE5YWUZO




    Week 5

    Translating ideas
    Develop fluency

    Homework
    • Watch 2 AE After Effects Tutorials (from the official Adobe website) and add 1 screenshot and 1 short description to the Class Notes Document (Instead of 2 AE tutorials, you can also watch 1 AE and 1 Figma-to-AE tutorial).
    • Create a draft animation of your project in which at least 2 elements are moving in order to communicate something
    • Read 2 theory pdfs by Anna & Renchao


    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Developing Fluidity in Sketching across materials, static and time-based
    3. Project Work (with After Effects Tutorial)
    4. Project Work (with After Effects Tutorial)

    Topics:
    • Paper Scribble
    • Sketching (as leaving out, abstracting, simplifying, reducing)
    • Adding scribble to animation
    • Adding text to animation
    • Adding photos to animation
    • Animate single vector point in path drawing (AE)


    Work samples shown here by Zoe, George (paper scribble), and by Lance, Connor, Judy, Emily, and Zoe (animation drafts)


    Week 6

    Overview of tools for experience designers
    Naming, understanding, and using tools

    Homework:
    • read theory pdfs before class
    • watch 2 additional AE tutorials that you have not watched yet, and add key elements to the class document (you can also add this document to find inspiration from other class members)
    • individual project: revise animation, make big progress. Make it longer, better, more expansive. Add at least 3 new elements from this list: text/annotations/keywords, arrows, lines, background color boxes, backgrounds, photos, photorealistic textures, animated textures, etc.). Respond to the feedback you have received. Goal: your animation should communicate 1 aspect of your project really well, without you having to explain it (the animation should be able to stand on its own and communicate to someone who doesn’t know your project at all). When you use that, use the scribble excercise from yesterday's class as a tool to decide which elements can be left out, which ones can be abstracted, and which ones need be shown in fullest detail?

    Today:
    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Workbook: Tools for Designers
    3. Workbook: Tools for Designers
    4. Animation review (homework review) & next steps; info about next week’s presentation

    Watched + discussed Adobe After Effects tutorials: 
    what is possible with AE, Bring your brand to life, Draw complex shapes, Keyframe animateion, Add realism with motion blur



    Image compositions shown here made by Marcel (water melon), Zoe, Rondo, Leo, Renchao, Kane, Emily


    Week 7

    Student presentations, with feedback by several (external) reviewers:

    Exhibition-style showcase of your project: you are not allowed to talk, rather you are required to show a video that can speak for itself. Video duration requirement: between 30 seconds and 2 minutes. Resolution: HD 1080p (16:9). With or without sound. The video should include, and combine, many elements of the following: title of project, student name, project field (example: product design, UX design), text, text annotations, keywords, drawing, animation, photo elements, etc. Use and apply all the skills learned in this class.

    This is a first draft of your “Large Project” for this course:

    Prompt: Each individual plans, realizes, presents and submits a large-scale design project: a larger ‘plan’ of an experience chosen by the individual, which involves different digital media, formats, forms, and styles to communicate the experience idea to a public that doesn’t know about the experience. The most common deliverables will be a video presenting, or being, the plan, along with a written report. Alternative deliverables are accepted if they are discussed ahead of time with the instructor (potentially VR projects, apps, series of drawings, and many others are accepted if there is a strong reason why they cannot be presented within a video).
    The progress of this large project will be presented to the class in an interim presentation (week 7) and a final presentation.


    Excerpts from Lance, Anna, Hanna, and Mia



    Week 8

    Learning from feedback:
    Presenting ideas

    Homework for today: prepare systematic responses and priorities to feedback received last week at the presentation. Come to class prepared with a list and prepared to discuss your list. 

    Today:
    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Case studies (Bose + Samsung), feedback on technicality that should be shown, input: Chapter 6.5, “The Review” (presenting ideas) – elevator pitch. Exercise: write someone else’s elevator pitch.
    3. Project work & tutorial meetings
    4. Project work & tutorial meetings



    Homework for next week:
    • Revise your video / extend it accoring to the feedback as discussed today. For many of you it involves shooting video, creating new animation parts, adding technical details, and illustrations. For some it requires to add sound. Upload the new version before the next class start to the folder "week 9 revisions"
    • Write a very short, but good, elevator pitch. It has four parts, according to our theory book. Upload your version before the next class start to the folder "week 9 elevator pitch".
    • Read the two pdf theory summaries that will be circulated next week before class.



    Week 9

    Analyzing experiences
    In-depth analysis of case studies

    Industry case studies, industry standard for product design, experience design, and animation showreels.

    Today:
    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Case studies: analyses of deep research in design that involves plans, sketches, illustrations, annotations, animation
      Discuss homework, elevator pitch, revisions
    3. Project work & tutorial meetings
    4. Project work & tutorial meetings



    Week 10

    Planning, combining, recombining, remixing
    Moving back and forth between plan and “thing”


    Works shown by Zoe, Emily, Renchao, George, Rondo, Lance

    1. Theory Presentations (examples, case studies, classroom discussion)
    2. Work on sketching/planning: annotating photo-realistice printed images by hand
    3. Project work & Tutorial meetings
    4. Project work & Tutorial meetings



    Week 11 (!)

    Watch out: no class on May 1st, but class is postponed to Saturday May 11th
    Communication skill: imagine your audience
    Present your work


    Week 12 - A (!) Wednesday

    Class on Wednesday + Saturday this week
    Critical thinking
    Text writing



    1. Guest Talk: Xuenan CAO on AI Extrapolation
    2. Project work & Tutorial meetings & Workshops as needed
    3. Project work & Tutorial meetings & Workshops as needed
    4. Workshop: Sound Design for Concept Design Videos







    Week 12 - B (!) Saturday

    Class on Wednesday + Saturday this week

    Work shown by Zoe, Judy, Rondo, and Marcel

    1. Guest presentation: portfolio, video, thesis project
    2. Project work & Tutorial meetings & Workshops as needed (3d work Rhino/Blender/AE/twinmotion)
    3. Project work & Tutorial meetings & Workshops as needed
    4. Project work & Tutorial meetings & Workshops as needed

    Helpful AE tutorials on 3d work:






    Week 13

    More tools: survey of advanced tools for experience designers


    Work shown by George and Leo



    Week 14

    Coursework submission, with classroom discussion

    Work shown by all students in DS366 (2024), photos by BAI Yiyuan.



    Week 15

    Composing experiences: moving between making and analyzing

    1. Practice for exam, repeat content
    2. Practice for exam, repeat content
    3. Project work, prepare for submission, instructions for submission
    4. Project work, prepare for submission, instructions for submission



    Week 16

    Work documentation & submission


    The deadline for the final submission is Thursday, June 6th, 23:59 (midnight). After that, the normal SUSTech School of Design late-policy applies, with deductions for every day, and after 5 days late, it's a fail (F). What exactly to submit: see at the bottom under “Coursework Submission”

    1. Exam practice and repetition of content
    2. Course evaluation
      https://forms.office.com/r/f6nrTdHvPg
    3. Project work, project submissions
    4. Project work, project submissions



    Weeks 17/18 - Exam Weeks

    Week 18: Wednesday, June 19, 2:00 - 4:00pm



    Core Readings


    • W. Buxton, Sketching User Experiences: getting the design right and the right design. Amsterdam Boston: Elsevier/Morgan Kaufmann, 2007.    
    • S. Greenberg, S. Carpendale, N. Marquardt, and B. Buxton, Sketching User Experiences. The Workbook. Elsevier, 2012. doi: 10.1016/B978-0-12-381959-8.50001-8.  

    Readings: Choice for Student Theory Presentations


    • Berridge, G. Events Design and Experience. (Butterworth-Heinemann, Oxford ; Burlington, MA, 2007).        
    • Bowles, C. & Box, J. Undercover User Experience: Learn How to Do Great UX Work with Tiny Budgets, No Time, and Limited Support. (New Riders, Berkeley, CA, 2011).        
    • Ciuccarelli, P., Lupi, G. & Simeone, L. Visualizing the Data City: Social Media as a Source of Knowledge for Urban Planning and Management. (Springer International Publishing, Cham, 2014). doi:10.1007/978-3-319-02195-9.    
    • Chapman, J. Emotionally Durable Design: Objects, Experiences and Empathy. (Routledge, 2015). doi:10.4324/9781315738802.
    • Conan, M. Landscape Design and the Experience of Motion. (Dumbarton Oaks Research Library and Collection, 2003).        
    • Participatory Design Theory: Using Technology and Social Media to Foster Civic Engagement. (Routledge, London New York, 2019).        
    • Mapping the Unmappable?: Cartographic Explorations with Indigenous Peoples in Africa. (transcript publishing, Bielefeld, 2021).        
    • Dohr, J. H. (Author). Design Thinking for Interiors : Inquiry, Experience, Impact.        
    • Dunne, A. Hertzian Tales: Electronic Products, Aesthetic Experience, and Critical Design. (MIT Press, Cambridge, Mass, 2005). 
    • Impett, Jonathan: Sound Work.Composition as Critical Technical Practice. artisticresearchreports.blogspot.com (2022).        
    • Jason Farman. Information Cartography: Visualizations of Internet Spatiality and Information Flows. in Composing(media) = composing(embodiment): bodies, technologies, writing, the teaching of writing (eds. Arola, K. L. & Wysocki, A. F.) (Utah State University Press, Logan, Utah, 2012).        
    • King, D. L. Designing the Digital Experience: How to Use Experience Design Tools and Techniques to Build Websites Customers Love. (CyberAge Books/Information Today, Medford, N.J, 2008).   
    • Koolhaas, R. Delirious New York. (Monacelli Press, New York, New York, 1994).     
    • Llano, S. Mapping street sounds in the nineteenth-century city: a listener’s guide to social engineering. 20 (2018).        
    • Lombardi, V. Why We Fail: Learning from Experience Design Failures. (Rosenfeld Media, Brooklyn, N.Y, 2013).        
    • Design, User Experience, and Usability. Theory, Methods, Tools and Practice: First International Conference, DUXU 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part II. vol. 6770 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2011).        
    • McMurray, P. Ephemeral cartography: on mapping sound. SOUND STUDIES 34 (2018).        
    • Ouzounian, G. Mapping the Acoustic City: Noise Mapping and Sound Mapping. in Stereophonica: sound and space in science, technology, and the arts (The MIT Press, Cambridge, Massachusetts, 2020).        
    • Robinson, S., Marsden, G. & Jones, M. There’s Not an App for That: Mobile User Experience Design for Life. (Elsevier/Morgan Kaufmann, Amsterdam ; Boston, 2015).        
    • Simon, M. Storyboards Motion in Art.        
    • Tabački, N. Consuming Scenography: The Shopping Mall as a Theatrical Experience. (Methuen Drama, London New York, 2020).        
    • Ward, P. Picture Composition for Film and Television. (Focal Press, Oxford ; Boston, 2003).        
    • User Experience Re-Mastered: Your Guide to Getting the Right Design. (Morgan Kaufmann Publishers, Burlington, MA, 2010).        
    • Xenakis, I. et al. From Xenakis’s UPIC to Graphic Notation Today. (Hatje Cantz, Berlin, (Germany), 2020).        
    • Yu, J. Shopping Experience: Store & Showroom. (Design Media Publishing, 2011).  


    Assessment

    • Attendance 10%
    • Final Exam 20%
    • Coursework 70% (33,3% for w14 final classroom presentation, 33,3% for w16 final submission, 33,3% for all work submitted regularly during the semester, every week, to the course folder, to show the progress)

    Attendance

    Attendance: 10%
    • 100 points for never late and never missing a class
    • 100 points for being up to 4 times late (up to 50min), OR for missing 1 entire class
    • 50 points for missing 1 entire class and being up to 4 times a little late (up to 50min)
    • 50 points for never missing a class, and being up to 8 times a little late (up to 50min)
    • 0 points for missing 2 or more classes
    • 0 points for being late for more than 8 times
    *all late or missing when unexcused. If excused with a reason, it doesn’t count toward the attendance grade.


    Large Project

    Prompt: Each individual plans, realizes, presents and submits a large-scale design project: a larger ‘plan’ of an experience chosen by the individual, which involves different digital media, formats, forms, and styles to communicate the experience idea to a public that doesn’t know about the experience. The most common deliverables will be a video presenting, or being, the plan, along with a written report. Alternative deliverables are accepted if they are discussed ahead of time with the instructor (potentially VR projects, apps, series of drawings, and many others are accepted if there is a strong reason why they cannot be presented within a video).
    The progress of this large project will be presented to the class in an interim presentation (week 7) and a final discussion/presentation (week 14).


    Coursework Submission

    • Video CN+EN: open duration, with or without sound, format: HD 16:9
    • Text Document CN+EN: product name, 1-line description, product slogan, design area (“UX Design”, ...), Name CN+Pinyin, course number+name, instructor name, for FYP people: supervisor name, abstract (everything in double language, open length)
    • 20 Images: a series of portfolio-style professional images, in high quality and high resolution, they must include at least 4 different styles/formats/materials that we have learned in this course; can include your choice of: pencil drawings, scribbles, photographs, photographs with annotations, digital illustrations, mixed pencil/digital illustrations, graphics, schematics, plans, scores, maps, flowcharts, storyboards, visualizations, sound visualizations, 3d renderings, screenshots of 3d models, screenshots from your video, screenshots from process, screenshots from software that you use (show also app interface around your work

    Important note: for FYP people, the materials submitted for DS366 can be (should be) identical with what you submit for the FYP (it is expected that you present materials in DS366 that are made for your FYP).


    Final classroom discussion in Week 14

    This will be an exhibition-style final showcase of your work based on a table-top. Each student is expected to show the coursework submission on a table-top layout (video playing on a computer screen, text and images laid out printed on paper on the table. Speaking is not allowed, the project is expected to speak for itself and be self-explanatory).


    Student Theory Presentations

    Each student sings up for a presentation of a theory topic. Each topic is based on an academic book or article on different forms of plans, experiences, time-based media, and experience composing. The prompt is for each individual in the class to choose a book/article, understand it thoroughly, and summarize the key take-aways in a short presentation for the peers. If it’s a book with many chapters or different essays, it’s better to choose one of them. The focus should be on how do we plan a time-based experience with visual presentation? How do plans and sketches look like? Is is the plan used in communicating the experience? And how does the plan/sketch feed back into the experience? Presentations are max 10 minutes, with a following 10 minutes for in-class discussion, moderated by the student.

    1. from the reading, choose the most important text parts and images (roughly 3 pages of text from the book, and 5 images from the book)
    2. create a handout for your peers with the choosen text and images (name sources and page numbers from original, and don’t change the text). If you add your own text to it, make sure you label clearly which is your own, and which is from the book.
    3. create a Powerpoint (or pdf) slide show that you will use to present the key ideas from the reading to the class
    4. present for max 10 minutes.
    5. after that, lead a class discussion (another 10 minutes). The best is to prepare 3-4 good questions that help you be a moderator.

    Submissions:

    • Until Friday evening 18:00: briefly discuss your choice of text with the instructor (email)
    • Upload handout for class: until Sunday night 23:59, to the class folder
    • Upload slides: until 14:00 on class day (Wednesday)


    Final Exam

    One part of the written exam is based on all the summaries of readings (academic books and articles) that have been presented in the student theory presentations. The second part is based on the summaries of the core class readings, presented by the instructor. The third part of the written exam is software knowledge of the tools taught and used in this class (primarily Adobe After Effects and Design Thinking).
    [Details will be added later]



    Classroom Policy

    After each class, the classroom has to be cleaned up properly. Everyone is responsible for their computer and their desk, chair, and the floor around them. Remove any food waste, any trash. Make order on the computer, on the desk. If the keyboard/mouse are low on battery, put them on charge. Backup your files privately, put the chair nicely to the desk.
    No eating during class time. Drinking is always allowed. The breaks can be used for eating.

    Equipment and Device Policy

    Technical devices are here to be used by the members of this class. They can be used freely during class time, and outside of class time, they can be either accessed in the classroom, or some of them can be checked out and can be taken with you. TA Jiayi is responsible for the device checkout.

    Academic Integrity

    https://msagesser.github.io/ds226-2023/SD-late-policy.pdf
    Sound Studies Group
    SUSTech School of Design    All content © 2023-2024 Sound Studies Group