Sound Studies Group





This is the Sound Studies Group at the SUSTech School of Design, Shenzhen. It is led by Dr. Qiushi Xu and Dr. Marcel Zaes Sagesser. Our research focuses on sound in relationship with technology, the environment, and human listening. It includes sound studies, philosophy of science and technology, HCI, and creative arts production. The output of this team is published in prestigious international journals, at domestic and international conferences, and at international art exhibitions as well as concert venues.


→  Members
→  Announcements & News

DS345 Sound Design 声音设计

Fall 2023


Prof. Marcel Zaes SAGESSER
Office hours Marcel: Friday 1:00 - 3:00pm, office, 3rd floor (or by appointment via email)
TA: Jiayi WEN

64 hours (32 lecture hours, 32 practice hours)

Class list
Equipment & Device Checkout List
Resources (protected folder)
Ongoing Class Notes
TCL P73 Series Home Theatre Soundbar
P5.js
Adobe XD - all tutorials
Adobe XD - sound prototyping I
Adobe XD - sound prototyping II
Figma - sound prototyping
Resource for waveform + spectrogram creation

Course Description

This course takes the students on a journey into the realm of the sonic. As something that is quite always around us and yet often remains underinterrogated, students learn about the emotional, narrative, social, and informative potential that sound has in contemporary media applications and beyond. This course focuses on sound as a malleable, raw material that we can craft, and that we can craft with – that is, sound as an opportunity for design. Students will learn to generate sound from scratch with synthesis, and by so doing, will develop critical listening skills, a vocabulary to analyze sound & affect, and will sharpen their skill to imagine auditory worlds for their future projects. The gained skill will help students more critically design intangible, socially engaging experiences in their life as designers. Beyond understanding the basic principles of acoustics, sound generation, composition, and audio editing, this course also surveys contemporary opportunities for the delivery of sound in digital media. The students will engage in English-language classroom discussion, will work both individually and in groups, and they will develop, produce, and deliver a large-scale sound design project that is relevant for experience design. This course might include field trips and/or collaborations with external partners depending on availability. Lecture and activity schedules are indicative and may be adjusted. Skills: composition, sound generation, editing.

Learning Outcomes

1.    Demonstrate skill in sound composition and sound editing
2.    Analyze, identify, and critically describe sound design opportunities within experience design
3.    Present original research on the social and cultural consequences of sound
4.    Realize and present an original high-quality creative project in sound design



Weekly Plan


1

Lecture (4 hours)
Introduction to sound design: what can sound do within experience and industrial design?
Welcome, overview of the course content, practice project, and the expectations about academic, ethical and creative rigor
  1. Course intro, course policies, collaboration partner
  2. Analyze sound examples / Reading Case p. 1-54, Intro + Chapter 1   
  3. Discussion + practice: recording
  4. Sound editing and discussion: what can sound do?

Three descriptors of sound?
How do we learn best to make sound?
What can sound do in design?
What kinds of sound design are there?
Three most fascinating words to describe sound?

Tech:

2

Lecture(4 hours)
Theory of acoustics and listening: what is sound, and how do we perceive it?
Basic knowledge, skill, and keywords, introduction to sound editing

  1. Analysis, sound making
  2. making
  3. Amber/Case, Chapter 3, pages 86-111
    Collins, Studying Sound, pages 50-80
  4. making

What could this product be?
How are the sounds technically made?

New Readings:
  • Collins, Karen. Studying Sound: A Theory and Practice of Sound Design. Cambridge, Massachusetts: The MIT Press, 2020.


3 (Sunday Oct 8)

Lecture and Practice (4 hours)
In the studio: sound recording
The principles of microphones, loudspeakers, storage, recording techniques, norms, and applications 
Small Assignment #1 is due: Submit to Blackboard before Sun Oct 8, 7:50am

  • Small Assignment #1
    Individual Work
  • Use VCV Rack and Adobe Audition
  • Create a generative sound design work that remains interesting for a duration of 2 minutes. “Interesting” means that some elements must be constantly changing (fast oder very slow, or both).
  • It should have at least 3 layers of sound (3 Oscillators / VCO).
  • It should use filters (VCF)
  • It should use time control (Step Sequencer SEQ and/or Low Frequency Oscillator LFO)
  • All sounds must be made with VCV Rack 2.
  • The sounds can be assembled in Adobe Audition. Audition can also be used to add additional reverb, or effects such as EQ, panning, automation.
  • Add a diagram (by hand or by computer) that shows the logic of your sound design.
  • Technical information: upload one stereo audio file (Left+Right) that is roughly 2 minutes long (mp3, wav, aif). Add your diagram (pdf, jpg, png, tif, or similar).  
  • Also attach the VCV Rack project file (.vcv), and a screenshot of your VCV Rack project that shows everything in a way that can be reproduced (pdf, png, etc.)


  1. Recap / Forecast.
    Intro Sound as Material
    Intro Sound Branding
    Analysis: branding examples
  2. Group making
  3. Filimowicz, Chapter 4, “Sketching Sonic Interactions,” p. 79-101
  4. Intro microphones, recording, loudspeakers, storage, techniques, applications
    Practice: create branding study, mixing synthesis, recording, and samples

New readings:
  • Filimowicz, Michael. Foundations in Sound Design for Embedded Media: A Multidisciplinary Approach. Routledge, 2020.

Resources:
https://freesound.org


4 (Oct 13)

Lecture and Practice (4 hours)
In the machine: sound synthesis
The principles of generating sound from scratch; using sound as a malleable raw material to craft with; sound in the machine as an opportunity for interaction
DS Talk: TCL Design Innovation Center (DIS) - project brief

  1. Review group work: Audio Branding Sketch (what is sketching?)
  2. Practice: continue sound synthesis, and editing: bringing together the elements - quick intro to interaction in sound
  3. Guest lecture: TCL
  4. Guest lecture: TCL

Homework for today: 
Make a sketch for an audio branding for a brand (choose any existing brand, or invent a fictional brand). Use one audio recording (self-made), one synthetic sound (self-made in VCV Rack), and one downloaded sample (freesound.org). You can put the three together with Adobe Audition. Compose an audio brand identity (or, and “audio logo) for your brand. Work in your groups. Please bring the result to class tomorrow, at the beginning, we will listen to the result of every group.





5

Lecture and Practice (4 hours)
Opportunities for sound in experience design
Survey of sound design applications, workshop on identifying and developing new opportunities, project work, background research, idea development
DS Talk: 吴圳 Zhen Wu (Yoyo) + 杨星星 Xingxing Yang - Designing for VR & VR Sound

  1. Intro to Interactivity, then: Project work in groups: Start conceptualizing (1) product opportunities and (2) “Nature-Inspired Sound”
  2. Project work in groups: Present your first ideas of sound design opportunities to the class.
  3. Guest lecture: Unity + sound interaction
  4. Guest lecture: Unity + sound interaction






6

Lecture and Practice (4 hours)
Sound and affect: the social and cultural consequences of sound
Theory of affect; sound as emotional, narrative, informative, and ambience. Theory of socio-cultural context of sound-making; project work, background research
DS Talk: 项湫涯 XIANG Qiuya (Angela) - Urban Sound Design in Hong Kong

  1. Audio Branding: read + take notes: Amber Case, Chapter 5, “Sound and Brand,” p. 127-141
  2. Final Project work: socio-cultural context of sound design (Case, p. 183)
  3. Guest lecture: Soundscape Design of Public Pedestrian Space
  4. Guest lecture: Soundscape Design of Public Pedestrian Space






7

Lecture and Practice (4 hours)
Composing sound is composing time: imagining auditory worlds
Conceptualizing timelines, perception of time, advanced audio composition: integration, collaging, cutting, transitioning, decontextualizing, working with depth, arranging time, and creating original auditory worlds

Homework for today: meet in your project team, and make progress on your final project. Finish Main project parts 1-2, as well as Project components 1-2 (see at the bottom of this website, under “Final Project”). Please bring these 4 finished parts to the class on Friday, Nov 3, 8:00am. We will discuss your results in class.

https://forms.microsoft.com/r/gVdVSbAxGm

  1. Introduction to Final Project Assignment
  2. Group Meetings / Project Work
  3. Group Meetings / Project Work
  4. Recap + audio editing / transitions / synthesis


8

Lecture and Practice (4 hours)
Advanced time structures: non-linearity and interactivity
Conceptualizing advanced timelines; non-linear, cyclical, interactive, and generative structures with their social and cultural consequences for users; project work, interim review preparations

Homework for today: meet in your project team, and make progress on your final project. Create drafts for your sounds (at least 2, a long and a short one, potentially more). Please bring these 2 sounds to class on Nov 10, 8:00am.

  1. Crafting sound: synthesis
  2. Crafting sound: recording
  3. Crafting sound: advanced editing
  4. Group Meetings / Project Work / Discuss Interim Review

9

Interim Review (4 hours)
Presentation and critique of background research and project drafts
(for detailed description, scroll all the way down)

10 (no class: SUSTech sports day)

Lecture and Practice (4 hours)
Design process: responding to critique and feedback
After the critique is before the critique: learning to respond to critique; rethinking the projects; project work, software and tech tutorials as needed by projects


11

Lecture and Practice (4 hours)
Crafting sound II: advanced synthesis, recording, sampling, and editing techniques
Sound as a material to create auditory worlds with; perfecting concepts and ideas in realization; project work; software and tech tutorials as needed by projects
Small Assignment #2 is due

Homework: (Due on  7:59am,  December 1)
  1. Respond to the feedback that you received from Prof. Enza Migliore, Prof. Lei, and myself
  2. Update all your materials. Define your usage scenario better, narrower. Define your user better. Polish your visualizations. Polish your slides, make them better
  3. Think about the story telling, and the order of your slides. Start with Lit Review, the conclusions of which lead to case studies, lead to your sound design concept
  4. Almost all groups don’t have a strong ’Sound Design Concept’ (except for Mia, Ricy, Emily). This slide should talk about how your research and your ideas are going to be realized in sound. Add this slide. 
  5. All groups: you all need to do more sound sketching. Make a variety of different sound. This process should be very modular. Don’t bring only finished sounds to class. It’s better to bring simple sound elements, parts, components, things that you made in VCV Rack, found online, recorded yourselves, maybe an also some voice recording / slogan. Make a lot of different elements that we can then combine in different ways in the next step. The process of sound design is very modular: we work with individual components that we combine in ever new ways to achieve variety.
  6. Many of your sound sketches don’t sound “modern” enough for a tech brand like “TCL.” In general, you all need to think about how to make it sound more “modern”, more “technological”, more “digital”.
  7. Start thinking about the last part of the final project: presenting your project in a video. Develop some ideas what you want to do for the video, how to best present your sound design / your usage scenario in a video.

 

12

Lecture and Practice (4 hours)
Creating space: sonic architectures and post production
When time becomes spatial: thinking time-based sound design as sonic architecture; learning the techniques of spatial processing; filtering, panning, layering, automating volumes, depth, compressing, and mixing sound productions; project work 
Small Assignment #2 is due

13

Final Review (4 hours)
With internal & external guests

14

Lecture and Practice (4 hours)
How to present something that is sonic: writing about and representing sound
Project work, internal feedback, and learning to write and present sound in a report and in a presentation

15

Practice (4 hours)
Tutorial & Presentation Skill
Project work, internal feedback, practice of presentation skill

16 (Jan 5)

Practice (4 hours)
Presentation and critique of final projects and project reports; submission of projects and project reports

17/18 (Jan 8-19)

Final Project Submission (Monday Jan 8)
Exam




Readings

  • Case, Amber, and Aaron Day. Designing with Sound: Fundamentals for Products and Services. 1st edition. Beijing ; Boston: O’Reilly Media, 2018.
  • Collins, Karen. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. MIT Press, 2008.
  • Collins, Karen. 2013. Playing with Sound: A Theory of Interacting with Sound and Music in Video Games. Illustrated edition. Cambridge, Massachusetts: The MIT Press.
  • Collins, Karen. Studying Sound: A Theory and Practice of Sound Design. Cambridge, Massachusetts: The MIT Press, 2020.
  • Filimowicz, Michael. Foundations in Sound Design for Embedded Media: A Multidisciplinary Approach. Routledge, 2020.
  • Filimowicz, Michael. Foundations in Sound Design for Interactive Media: A Multidisciplinary Approach. Routledge, 2019.
  • Horowitz, Steve, and Scott R. Looney. 2014. The Essential Guide to Game Audio: The Theory and Practice of Sound for Games. CRC Press.
  • Scott-James, Kahra. Sound Design for Moving Image: From Concept to Realization. London ; New York: Bloomsbury Academic, 2018.
  • Sinclair, Jean-Luc. Principles of Game Audio and Sound Design. 1st edition. New York, NY: Focal Press, 2020



Assessment

  • Attendance 10%
  • Small Assignments (2) 20%
  • Final Exam 20%
  • Final Project 50%



Attendance

Attendance: 10%
  • 0 min late = on time
  • up to 30 min late = a little late
  • more than 30 min late = late (the half day counts as absent)

Explanation
  • on time and no more than four times a little late = 10%
  • more than five times a little late = 5%
  • once late (= half day absent) = 5%
  • twice or more late (= two or more half days absent) = 0%


Final Project

Each group creates a large-scale design project in sound design. The task is to use TCL’s P73 Soundbar as an opportunity to explore the use of sound design for this product. We are thinking the “product” (P73) not as an isolated product that makes sound, but rather, as a flexible device that exists in an environment that is characterized by its social, cultural, and spatial specifics. We understand the product from start to end: from how it’s advertized, marketed, exhibited, and finally, how it’s applied/used. The task is to use, and further develop TCL’s “Nature-Inspired Sound,” and to apply it for specifically a young target audience. 

Main project parts
  1. choose a (1) usage scenario
  2. research: identify design opportunities in your scenario
  3. design: create a sound design proposal for your scenario, including a working (virtual) prototype (P5.js)
  4. realize: create the sound elements, and visualize their application in a presentable, visual way
  5. documentation: create a short video presentation (2-4 min) and a pdf project report (8-16 pages)

Project components
  1. Visualization of your usage scenario (3d model or scene)
  2. Desk research (at least 3 academic references, and 2 case studies)
  3. Branding: how to render the brand “TCL” perceivable through your design?
  4. Concept: how to tailor  TCL’s “Nature-Inspired Sound” to a young target audience?
  5. Sound Design: create at least 5 original elements of sound design for your project. These sounds could include, but are not limited to: sonic feedback on the device, like Bluetooth pairing or on/off sounds, switching between different modes; an audio logo / audio identity for the product; a soundtrack for product exhibitions at a fair; a soundtrack for a product ad; a sound design for a remote-control APP for the P73; and many more...
  6. The sound design components need to contain at least 1 self-made recording, 1 self-made VCV Rack sound
  7. An interactive prototype for your usage scenario (coded with P5.js)
  8. Documentation film (2-4min): the content/form is quite open, but it must contain your sound design elements as the main sounds for the video. It also must contain the visualization of your usage scenario, and it contains your version of “Nature-Inspired” for a young audience. It could use visualization of sound, it could use animation / 3d scenes, it could use real-world film scenes, or many other materials that help present the product in its social and spatial environment. The documentation film needs to be double language, English/Chinese.
  9. The report (a written document, 8-16 pages pdf), is written in double language, English/Chinese. It contains your research, litearture, case studies, your concept, your 3d sketch of the P73, the usage scenario, still images from your design process, sketches, mind maps, drawings, diagrams, a list of your sound materials, and visualizations of your sound design.


Interim Review

The interim review is the first step toward the final project. The presentation needs to include very good drafts of Main project parts 1-4, as well as very good drafts of the Project components 1-7.


Classroom Policy

After each class, the classroom has to be cleaned up properly. Everyone is responsible for their computer and their desk, chair, and the floor around them. Remove any food waste, any trash. Make order on the computer, on the desk. If the keyboard/mouse are low on battery, put them on charge. Backup your files privately, put the chair nicely to the desk.
No eating during class time. Drinking is always allowed. The breaks can be used for eating.



Equipment and Device Policy

Technical devices are here to be used by the members of this class. They can be used freely during class time, and outside of class time, they can be either accessed in the classroom, or some of them can be checked out and can be taken with you. TA Jiayi is responsible for the device checkout.



Academic Integrity

https://msagesser.github.io/ds226-2023/SD-late-policy.pdf
Homework: 
Sound Studies Group
SUSTech School of Design
All content © 2023 Sound Studies Group