Sound Studies Group


→ Activities
-> Publications
→ Members
→ Gallery
→ Contact

This is the Sound Studies Group at the SUSTech School of Design, Shenzhen. It is led by Dr. Qiushi Xu and Dr. Marcel Zaes Sagesser. Our research focuses on sound in relationship with technology, the environment, and human listening. It includes sound technology, sound studies, Human Computer Interaction (HCI), sound design, philosophy of science and technology, spatial audio, interactive audio, and creative media. The output of this team is published in prestigious international journals, at domestic and international conferences, and at international exhibitions and concert halls.

#Sound Studies
#Creative Media
#Digital Media Arts
#VR + Sonic Architectures
#Interaction Design
#Sound Design
#Auditory Interaction
#Human Computer Interaction HCI


We are continuously looking for exceptional students for short-term or long-term internships and visiting student positions in our group. Please send us your CV and your portfolio if you are interested.

DS345 Sound Design 声音设计

Fall 2023


Prof. Marcel Zaes SAGESSER
Office hours Marcel: Friday 1:00 - 3:00pm, office, 3rd floor (or by appointment via email)
TA: Jiayi WEN

64 hours (32 lecture hours, 32 practice hours)



Class list
Equipment & Device Checkout List
Resources (protected folder)
Ongoing Class Notes
Text Writing Workshop
TCL P73 Series Home Theatre Soundbar
P5.js
Adobe XD - all tutorials
Adobe XD - sound prototyping I
Adobe XD - sound prototyping II
Figma - sound prototyping
Resource for waveform + spectrogram creation
Film sound clichés (stereotypes)
Learning Music with Ableton
Reaper Audio Production (freeware)
https://capitalizemytitle.com/

Course Description

This course takes the students on a journey into the realm of the sonic. As something that is quite always around us and yet often remains underinterrogated, students learn about the emotional, narrative, social, and informative potential that sound has in contemporary media applications and beyond. This course focuses on sound as a malleable, raw material that we can craft, and that we can craft with – that is, sound as an opportunity for design. Students will learn to generate sound from scratch with synthesis, and by so doing, will develop critical listening skills, a vocabulary to analyze sound & affect, and will sharpen their skill to imagine auditory worlds for their future projects. The gained skill will help students more critically design intangible, socially engaging experiences in their life as designers. Beyond understanding the basic principles of acoustics, sound generation, composition, and audio editing, this course also surveys contemporary opportunities for the delivery of sound in digital media. The students will engage in English-language classroom discussion, will work both individually and in groups, and they will develop, produce, and deliver a large-scale sound design project that is relevant for experience design. This course might include field trips and/or collaborations with external partners depending on availability. Lecture and activity schedules are indicative and may be adjusted. Skills: composition, sound generation, editing.

Learning Outcomes

1.    Demonstrate skill in sound composition and sound editing
2.    Analyze, identify, and critically describe sound design opportunities within experience design
3.    Present original research on the social and cultural consequences of sound
4.    Realize and present an original high-quality creative project in sound design



Weekly Plan


1

Lecture (4 hours)
Introduction to sound design: what can sound do within experience and industrial design?
Welcome, overview of the course content, practice project, and the expectations about academic, ethical and creative rigor
  1. Course intro, course policies, collaboration partner
  2. Analyze sound examples / Reading Case p. 1-54, Intro + Chapter 1   
  3. Discussion + practice: recording
  4. Sound editing and discussion: what can sound do?

Three descriptors of sound?
How do we learn best to make sound?
What can sound do in design?
What kinds of sound design are there?
Three most fascinating words to describe sound?

Tech:

2

Lecture(4 hours)
Theory of acoustics and listening: what is sound, and how do we perceive it?
Basic knowledge, skill, and keywords, introduction to sound editing

  1. Analysis, sound making
  2. making
  3. Amber/Case, Chapter 3, pages 86-111
    Collins, Studying Sound, pages 50-80
  4. making

What could this product be?
How are the sounds technically made?

New Readings:
  • Collins, Karen. Studying Sound: A Theory and Practice of Sound Design. Cambridge, Massachusetts: The MIT Press, 2020.


3 (Sunday Oct 8)

Lecture and Practice (4 hours)
In the studio: sound recording
The principles of microphones, loudspeakers, storage, recording techniques, norms, and applications 
Small Assignment #1 is due: Submit to Blackboard before Sun Oct 8, 7:50am

  • Small Assignment #1
    Individual Work
  • Use VCV Rack and Adobe Audition
  • Create a generative sound design work that remains interesting for a duration of 2 minutes. “Interesting” means that some elements must be constantly changing (fast oder very slow, or both).
  • It should have at least 3 layers of sound (3 Oscillators / VCO).
  • It should use filters (VCF)
  • It should use time control (Step Sequencer SEQ and/or Low Frequency Oscillator LFO)
  • All sounds must be made with VCV Rack 2.
  • The sounds can be assembled in Adobe Audition. Audition can also be used to add additional reverb, or effects such as EQ, panning, automation.
  • Add a diagram (by hand or by computer) that shows the logic of your sound design.
  • Technical information: upload one stereo audio file (Left+Right) that is roughly 2 minutes long (mp3, wav, aif). Add your diagram (pdf, jpg, png, tif, or similar).  
  • Also attach the VCV Rack project file (.vcv), and a screenshot of your VCV Rack project that shows everything in a way that can be reproduced (pdf, png, etc.)


  1. Recap / Forecast.
    Intro Sound as Material
    Intro Sound Branding
    Analysis: branding examples
  2. Group making
  3. Filimowicz, Chapter 4, “Sketching Sonic Interactions,” p. 79-101
  4. Intro microphones, recording, loudspeakers, storage, techniques, applications
    Practice: create branding study, mixing synthesis, recording, and samples

New readings:
  • Filimowicz, Michael. Foundations in Sound Design for Embedded Media: A Multidisciplinary Approach. Routledge, 2020.

Resources:
https://freesound.org


4 (Oct 13)

Lecture and Practice (4 hours)
In the machine: sound synthesis
The principles of generating sound from scratch; using sound as a malleable raw material to craft with; sound in the machine as an opportunity for interaction
DS Talk: TCL Design Innovation Center (DIS) - project brief

  1. Review group work: Audio Branding Sketch (what is sketching?)
  2. Practice: continue sound synthesis, and editing: bringing together the elements - quick intro to interaction in sound
  3. Guest lecture: TCL
  4. Guest lecture: TCL

Homework for today: 
Make a sketch for an audio branding for a brand (choose any existing brand, or invent a fictional brand). Use one audio recording (self-made), one synthetic sound (self-made in VCV Rack), and one downloaded sample (freesound.org). You can put the three together with Adobe Audition. Compose an audio brand identity (or, and “audio logo) for your brand. Work in your groups. Please bring the result to class tomorrow, at the beginning, we will listen to the result of every group.





5

Lecture and Practice (4 hours)
Opportunities for sound in experience design
Survey of sound design applications, workshop on identifying and developing new opportunities, project work, background research, idea development
DS Talk: 吴圳 Zhen Wu (Yoyo) + 杨星星 Xingxing Yang - Designing for VR & VR Sound

  1. Intro to Interactivity, then: Project work in groups: Start conceptualizing (1) product opportunities and (2) “Nature-Inspired Sound”
  2. Project work in groups: Present your first ideas of sound design opportunities to the class.
  3. Guest lecture: Unity + sound interaction
  4. Guest lecture: Unity + sound interaction






6

Lecture and Practice (4 hours)
Sound and affect: the social and cultural consequences of sound
Theory of affect; sound as emotional, narrative, informative, and ambience. Theory of socio-cultural context of sound-making; project work, background research
DS Talk: 项湫涯 XIANG Qiuya (Angela) - Urban Sound Design in Hong Kong

  1. Audio Branding: read + take notes: Amber Case, Chapter 5, “Sound and Brand,” p. 127-141
  2. Final Project work: socio-cultural context of sound design (Case, p. 183)
  3. Guest lecture: Soundscape Design of Public Pedestrian Space
  4. Guest lecture: Soundscape Design of Public Pedestrian Space






7

Lecture and Practice (4 hours)
Composing sound is composing time: imagining auditory worlds
Conceptualizing timelines, perception of time, advanced audio composition: integration, collaging, cutting, transitioning, decontextualizing, working with depth, arranging time, and creating original auditory worlds

Homework for today: meet in your project team, and make progress on your final project. Finish Main project parts 1-2, as well as Project components 1-2 (see at the bottom of this website, under “Final Project”). Please bring these 4 finished parts to the class on Friday, Nov 3, 8:00am. We will discuss your results in class.

https://forms.microsoft.com/r/gVdVSbAxGm

  1. Introduction to Final Project Assignment
  2. Group Meetings / Project Work
  3. Group Meetings / Project Work
  4. Recap + audio editing / transitions / synthesis


8

Lecture and Practice (4 hours)
Advanced time structures: non-linearity and interactivity
Conceptualizing advanced timelines; non-linear, cyclical, interactive, and generative structures with their social and cultural consequences for users; project work, interim review preparations

Homework for today: meet in your project team, and make progress on your final project. Create drafts for your sounds (at least 2, a long and a short one, potentially more). Please bring these 2 sounds to class on Nov 10, 8:00am.

  1. Crafting sound: synthesis
  2. Crafting sound: recording
  3. Crafting sound: advanced editing
  4. Group Meetings / Project Work / Discuss Interim Review

9

Interim Review (4 hours)
Presentation and critique of background research and project drafts
(for detailed description, scroll all the way down)

10 (no class: SUSTech sports day)

Lecture and Practice (4 hours)
Design process: responding to critique and feedback
After the critique is before the critique: learning to respond to critique; rethinking the projects; project work, software and tech tutorials as needed by projects

11

Lecture and Practice (4 hours)
Crafting sound II: advanced synthesis, recording, sampling, and editing techniques
Sound as a material to create auditory worlds with; perfecting concepts and ideas in realization; project work; software and tech tutorials as needed by projects
Design process: responding to critique and feedback
After the critique is before the critique: learning to respond to critique; rethinking the projects; project work, software and tech tutorials as needed by projects

  1. lecture/workshop: responding to student feedback and project needs
    including reading: Case Amber, Chapter 9 “Prototyping”, Horowitz Game Audio Book chapter 5 “Sound Design in Games”
    including 2 cases, layers of sound design, film sound clichés
  2. (continuation)
  3. group work + group tutorials
  4. group work + group tutorials


Small Assignment #2 is due

Homework: (Due on  7:59am,  December 1)
  1. Respond to the feedback that you received from Prof. Enza Migliore, Prof. Lei, and myself
  2. Update all your materials. Define your usage scenario better, narrower. Define your user better. Polish your visualizations. Polish your slides, make them better
  3. Think about the story telling, and the order of your slides. Start with Lit Review, the conclusions of which lead to case studies, lead to your sound design concept
  4. Almost all groups don’t have a strong ’Sound Design Concept’ (except for Mia, Ricy, Emily). This slide should talk about how your research and your ideas are going to be realized in sound. Add this slide. 
  5. All groups: you all need to do more sound sketching. Make a variety of different sound. This process should be very modular. Don’t bring only finished sounds to class. It’s better to bring simple sound elements, parts, components, things that you made in VCV Rack, found online, recorded yourselves, maybe an also some voice recording / slogan. Make a lot of different elements that we can then combine in different ways in the next step. The process of sound design is very modular: we work with individual components that we combine in ever new ways to achieve variety.
  6. Many of your sound sketches don’t sound “modern” enough for a tech brand like “TCL.” In general, you all need to think about how to make it sound more “modern”, more “technological”, more “digital”.
  7. Start thinking about the last part of the final project: presenting your project in a video. Develop some ideas what you want to do for the video, how to best present your sound design / your usage scenario in a video.

 

12

Lecture and Practice (4 hours)
Creating space: sonic architectures and post production
When time becomes spatial: thinking time-based sound design as sonic architecture; learning the techniques of spatial processing; filtering, panning, layering, automating volumes, depth, compressing, and mixing sound productions; project work 

Responding to student feedback: Understand the basics of music, harmony, rhythm.

  1. Introduce Final Presentations
  2. Intro to music, harmony rhythm
  3. Project workshop
  4. Project workshop


Small Assignment #2 is due today at 7:59am  to the course folder (will be time-marked)

  • Small Assignment #2
    Individual Work
  • Use Adobe Audition and any other tool of your choice
  • Use at least 3 different sound samples (they can by your own recordings, your own synthesis, or from online resources)
  • Combine these three into 1 single sound/experience
  • Task: create a sound that is “natural,” but doesn’t contain nature sounds
  • Apply all your advanced sound editing skills to create a smooth, beautiful sound design
  • length: should be between 10-30 seconds.
  • Upload: by Dec 8th, 7:59am, upload to the course folder “Assignment #2”.
  • Upload elements: 3 original sound files (clearly labelled), 1 final sound design (sound file, clearly labelled), 1 text file (.pdf) with a very short description: what are your original files? what is your idea? how did you do it?


13

Final Review (4 hours)
With guest reviewers

Final Presentations:
  • 5 min presentation per group (3-4 min of talking, 1-2 min of film/sound playing)
  • Slide structure: 1 intro slide (project title, subtitle, names), 1-2 usage scenario + design opportunities slides, 1-2 research slides (only best-of desk research + case studies), 1-2 concept slides (what is your main sound design idea? how do you integrate your usage scenarios with TCL’s concepts?), 1 design slide (describe your design project), 1 source slide (where are your materials from?), then: play film/documentary/sound without further comments.


14

Lecture and Practice (4 hours)
How to present something that is sonic: writing about and representing sound
Project work, internal feedback, and learning to write and present sound in a report and in a presentation

  1. What is a blurb/abstract/project text? Sound & Text workshop, analysis of examples
  2. Group work on text
  3. Project work on final report (+ individual meetings with Marcel)
  4. Project work on final report (+ individual meetings with Marcel)

15

Practice (4 hours)
Tutorial & Presentation Skill
Project work, internal feedback, practice of presentation skill

  1. project work: revise report, revise videos, revise EN+CN texts (Tutorials with Marcel)
  2. project work: revise report, revise videos, revise EN+CN texts (Tutorials with Marcel)
  3. project work: revise report, revise videos, revise EN+CN texts (Tutorials with Marcel)
  4. project work: revise report, revise videos, revise EN+CN texts (Tutorials with Marcel)


Homework (due at 7:59 am on December 29 to the course folder):
  • good English title and subtitle that gives the audience a full idea of your project.
  • same in Chinese: does not need to be a literal translation, but must work well in Chinese (Jiayi will workshop the Chinese titles with you).
  • good description text in English (English: ideally around150-200 English words or a little bit longer would be great, but, the shorter the better). You can say more with less.
  • good description text in Chinese (equivalent length, but shorter is better. Does not need to be a word-by-word translation, but a text that works very well.
  • All titles and texts, in both languages, need to be grammar corrected and spelling corrected. Titles and subtitles must use title case for capitalization (check them on https://capitalizemytitle.com/ or similar websites).
  • Upload your finished titles/subtitles/texts in EN+CN until Friday morning, Dec 29, 7:59 am, to the course folder (subfolder text workshop / draft 2) 


16 (Jan 5) - Last Teaching Week

Practice (4 hours)
Presentation and critique of final projects and project reports; submission of projects and project reports

  1. exam preparations and tutorials
  2. exam preparations and tutorials
  3. course feedback + evaluations
    https://tis.sustech.edu.cn/ -业务办理-评教任务-2023秋季学期学生评价任务。
  4. final project uploads


17 - Exam Week 1


Final Project Submission (Monday Jan 8th, 2024, 23:59)
 submit the following documents to the class folder. Create a subfolder with your team names. Your folder must contain:

  • Revised video (1-3 min), with revised soundtrack
    (if the video contains any language, add EN + CN)

  • Final report, pdf, 8-16 pages, is entirely written in double language EN/CN.
           it contains:
        - title page according to Marcel’s template: revised title + subtitle in EN/CN. Names in Chinese + Pinyin.
        - the second page contains your abstract text that we revised, in EN/CN
        - it’s based on your slides from the final presentation
        - format is open. Can be A4, A3, or Powerpoint 16:9 format. Can be landscape or portrait format. You choose.
        - style/fonts/sizes is completely up to you. Make it look professional
        - the report contains all your materials from the project and from the final presentation slides:
            
      your research, literature, case studies, your concept, your 3d sketch of the P73, the usage scenario, still images from your design process, sketches, mind maps, drawings, diagrams, a list of your sound materials (self recorded? self synthesized? from online sources), video sources, and visualizations of your sound design. 

    1. 5 high quality images (1+4), in high resolution. There’s 1 main project image for the project, and 4 secondary images. They need to be representative of your project. They can be high-resolution stills from your video, can be AI generated, or high-quality renderings from your 3d models and usage scenarios, or even diagrams/reseearch parts, as long as they represent your project. Upload them as high-resolution JPG, PNG, TIFF, or similar. -> the images will be used to present your projects in exhibitions and public School presentations.


    18 - Exam Week 2

    January 18, 14:00 - 16:00, DS345 final exam, Teaching Building Three, Room 302



    Readings

    • Case, Amber, and Aaron Day. Designing with Sound: Fundamentals for Products and Services. 1st edition. Beijing ; Boston: O’Reilly Media, 2018.
    • Collins, Karen. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. MIT Press, 2008.
    • Collins, Karen. 2013. Playing with Sound: A Theory of Interacting with Sound and Music in Video Games. Illustrated edition. Cambridge, Massachusetts: The MIT Press.
    • Collins, Karen. Studying Sound: A Theory and Practice of Sound Design. Cambridge, Massachusetts: The MIT Press, 2020.
    • Filimowicz, Michael. Foundations in Sound Design for Embedded Media: A Multidisciplinary Approach. Routledge, 2020.
    • Filimowicz, Michael. Foundations in Sound Design for Interactive Media: A Multidisciplinary Approach. Routledge, 2019.
    • Horowitz, Steve, and Scott R. Looney. 2014. The Essential Guide to Game Audio: The Theory and Practice of Sound for Games. CRC Press.
    • Scott-James, Kahra. Sound Design for Moving Image: From Concept to Realization. London ; New York: Bloomsbury Academic, 2018.
    • Sinclair, Jean-Luc. Principles of Game Audio and Sound Design. 1st edition. New York, NY: Focal Press, 2020



    Assessment

    • Attendance 10%
    • Small Assignments (2) 20%
    • Final Exam 20%
    • Final Project 50%



    Attendance

    Attendance: 10%
    • 0 min late = on time
    • up to 30 min late = a little late
    • more than 30 min late = late (the half day counts as absent)

    Explanation
    • on time and no more than four times a little late = 10%
    • more than five times a little late = 5%
    • once late (= half day absent) = 5%
    • twice or more late (= two or more half days absent) = 0%


    Final Project

    Each group creates a large-scale design project in sound design. The task is to use TCL’s P73 Soundbar as an opportunity to explore the use of sound design for this product. We are thinking the “product” (P73) not as an isolated product that makes sound, but rather, as a flexible device that exists in an environment that is characterized by its social, cultural, and spatial specifics. We understand the product from start to end: from how it’s advertized, marketed, exhibited, and finally, how it’s applied/used. The task is to use, and further develop TCL’s “Nature-Inspired Sound,” and to apply it for specifically a young target audience. 

    Main project parts
    1. choose a (1) usage scenario
    2. research: identify design opportunities in your scenario
    3. design: create a sound design proposal for your scenario, including a working (virtual) prototype (P5.js)
    4. realize: create the sound elements, and visualize their application in a presentable, visual way
    5. documentation: create a short video presentation (1-3 min) and a pdf project report (8-16 pages)

    Project components
    1. Visualization of your usage scenario (3d model or scene)
    2. Desk research (at least 3 academic references, and 2 case studies)
    3. Branding: how to render the brand “TCL” perceivable through your design?
    4. Concept: how to tailor  TCL’s “Nature-Inspired Sound” to a young target audience?
    5. Sound Design: create at least 5 original elements of sound design for your project. These sounds could include, but are not limited to: sonic feedback on the device, like Bluetooth pairing or on/off sounds, switching between different modes; an audio logo / audio identity for the product; a soundtrack for product exhibitions at a fair; a soundtrack for a product ad; a sound design for a remote-control APP for the P73; and many more...
    6. The sound design components need to contain at least 1 self-made recording, 1 self-made VCV Rack sound
    7. An interactive prototype for your usage scenario (coded with a tool of your choice, such as PPT, P5.js, Figma, Adobe XD, etc.) --> can be integrated with your video documentation (in which case it is not interactive)
    8. Documentation film (1-3 min): the content/form is quite open, but it must contain your sound design elements as the main sounds for the video. It also must contain the visualization of your usage scenario, and it contains your version of “Nature-Inspired” for a young audience. It could use visualization of sound, it could use animation / 3d scenes, it could use real-world film scenes, or many other materials that help present the product in its social and spatial environment. The documentation film needs to be double language, English/Chinese.
    9. The report (a written document, 8-16 pages pdf), is written in double language, English/Chinese. It contains your research, literature, case studies, your concept, your 3d sketch of the P73, the usage scenario, still images from your design process, sketches, mind maps, drawings, diagrams, a list of your sound materials, and visualizations of your sound design.


    Final Exam

    The written exam of DS345 is based on the following materials that were discussed in class:
    1. Audio Editing: software Adobe Audition skills
    2. Sound Synthesis: software VCV Rack skills
    3. Readings: all chapters that have been studied in class, see list underneath:

    【腾讯文档】DS345-2023 Exam Prepping
    https://docs.qq.com/doc/DSlZSbVdQZ2JRQktl

    Case, Amber, and Aaron Day. Designing with Sound: Fundamentals for Products and Services. 1st edition. Beijing ; Boston: O’Reilly Media, 2018.
    • Preface  (p. 6-38) - week 1
    • Chapter 1: Opportunities in Sound Design (p. 40-54) - week 1
    • Chapter 3: Adding Sound to Interactions (p. 86-111) - week 2
    • Chapter 5: Sound and Brand (p. 127-141) - week 6
    • Chapter 9: Prototyping (p. 191-202) - week 11

    Collins, Karen. Studying Sound: A Theory and Practice of Sound Design. Cambridge, Massachusetts: The MIT Press, 2020.
    • Chapter 2: Sound (p. 50-80, not the full chapter) - week 2

    Filimowicz, Michael. Foundations in Sound Design for Embedded Media: A Multidisciplinary Approach. Routledge, 2020.
    • Chapter 4: Sketching Sonic Interactions (p. 79-101) - week 4

    Horowitz, Steve, and Scott R. Looney. 2014. The Essential Guide to Game Audio: The Theory and Practice of Sound for Games. CRC Press. 
    • Chapter 5: Sound Design in Games (p. 73-93) - week 11



    Interim Review

    The interim review is the first step toward the final project. The presentation needs to include very good drafts of Main project parts 1-4, as well as very good drafts of the Project components 1-7.


    Classroom Policy

    After each class, the classroom has to be cleaned up properly. Everyone is responsible for their computer and their desk, chair, and the floor around them. Remove any food waste, any trash. Make order on the computer, on the desk. If the keyboard/mouse are low on battery, put them on charge. Backup your files privately, put the chair nicely to the desk.
    No eating during class time. Drinking is always allowed. The breaks can be used for eating.



    Equipment and Device Policy

    Technical devices are here to be used by the members of this class. They can be used freely during class time, and outside of class time, they can be either accessed in the classroom, or some of them can be checked out and can be taken with you. TA Jiayi is responsible for the device checkout.



    Academic Integrity

    https://msagesser.github.io/ds226-2023/SD-late-policy.pdf
    Homework: 

    Sound Studies Group
    SUSTech School of Design    All content © 2023-2024 Sound Studies Group