Keepsake
Keepsake is a memory-preserving device that utilizes Al-powered object recognition (LLaVA). It allows users to record personal stories linked to specific objects and replay them anytime. Imagine listening to your own recorded memories one year or even ten years later. What if you could revisit the stories embedded in your childhood belongings and share them with someone in the future?
Keepsake is a memory-preserving device that utilizes Al-powered object recognition (LLaVA). It allows users to record personal stories linked to specific objects and replay them anytime. Imagine listening to your own recorded memories one year or even ten years later. What if you could revisit the stories embedded in your childhood belongings and share them with someone in the future?
Keepsake is a memory-preserving device that utilizes Al-powered object recognition (LLaVA). It allows users to record personal stories linked to specific objects and replay them anytime. Imagine listening to your own recorded memories one year or even ten years later. What if you could revisit the stories embedded in your childhood belongings and share them with someone in the future?
Group / Individual
Group / Individual
Group / Individual
Group Project with Edha Singla and Gayatri Kalra
Group Project with Edha Singla and Gayatri Kalra
Group Project with Edha Singla and Gayatri Kalra
Location
Location
Copenhagen Institute of Interaction Design (Bergamo, Italy)
Copenhagen Institute of Interaction Design (Bergamo, Italy)
Copenhagen Institute of Interaction Design (Bergamo, Italy)
Course
Course
Tangible User Interface (Lecturers : Massimo Banzi, Jose Chavarria and Pierluigi Dalla Rosa)
Tangible User Interface (Lecturers : Massimo Banzi, Jose Chavarria and Pierluigi Dalla Rosa)
Tangible User Interface (Lecturers : Massimo Banzi, Jose Chavarria and Pierluigi Dalla Rosa)
Duration
Duration
Duration
2025. 01. 20 ~ 01. 31 (12 days)
2025. 01. 20 ~ 01. 31 (12 days)
2025. 01. 20 ~ 01. 31 (12 days)
How Keepsake works
How Keepsake works
How Keepsake works
Object Placement & Capture
The user places an object on Keepsake’s surface, where a built-in camera captures its image. This image is then processed through a locally hosted LLaVA model, extracting the object’s primary characteristics.
Memory Recognition
A custom-built web application analyses the captured image to determine if a memory is already linked to the object. This system ensures that the right narrative is paired with the right artifact.
Playback or Recording
If a memory exists, it plays automatically. If no memory is found, the device invites the user to record a new message.
Object Placement & Capture
The user places an object on Keepsake’s surface, where a built-in camera captures its image. This image is then processed through a locally hosted LLaVA model, extracting the object’s primary characteristics.
Memory Recognition
A custom-built web application analyses the captured image to determine if a memory is already linked to the object. This system ensures that the right narrative is paired with the right artifact.
Playback or Recording
If a memory exists, it plays automatically. If no memory is found, the device invites the user to record a new message.
Object Placement & Capture
The user places an object on Keepsake’s surface, where a built-in camera captures its image. This image is then processed through a locally hosted LLaVA model, extracting the object’s primary characteristics.
Memory Recognition
A custom-built web application analyses the captured image to determine if a memory is already linked to the object. This system ensures that the right narrative is paired with the right artifact.
Playback or Recording
If a memory exists, it plays automatically. If no memory is found, the device invites the user to record a new message.
< Sensor & Technology configuration >
< Sensor & Technology configuration >
< Sensor & Technology configuration >
Process
Process
How can we store our precious memories in everyday objects?
How can we store our precious memories in everyday objects?
How can we store our precious memories in everyday objects?
We all have objects that hold memories, and seeing those objects lets us imagine those precious moments. Ordinary things are special because they carry memories. We wanted to expand that experience a little further.
We all have objects that hold memories, and seeing those objects lets us imagine those precious moments. Ordinary things are special because they carry memories. We wanted to expand that experience a little further.
We all have objects that hold memories, and seeing those objects lets us imagine those precious moments. Ordinary things are special because they carry memories. We wanted to expand that experience a little further.
Ideated various interface layouts and form factors through low-fidelity prototyping
Ideated various interface layouts and form factors through low-fidelity prototyping
Ideated various interface layouts and form factors through low-fidelity prototyping
Tested pressure sensors for tray interaction triggers but pivoted to load cells (weight sensors) for superior signal stability.
Tested pressure sensors for tray interaction triggers but pivoted to load cells (weight sensors) for superior signal stability.
Tested pressure sensors for tray interaction triggers but pivoted to load cells (weight sensors) for superior signal stability.
Trigger: Arduino sends a signal via serial port when weight exceeds the threshold.
Analysis: Local Vision AI (via LM Studio) analyzes the object upon receiving the signal.
Action: Plays the corresponding MP3 from local storage if the object is recognized.
Recording: If no file exists, the user records and saves a new MP3 via the web interface.
Trigger: Arduino sends a signal via serial port when weight exceeds the threshold.
Analysis: Local Vision AI (via LM Studio) analyzes the object upon receiving the signal.
Action: Plays the corresponding MP3 from local storage if the object is recognized.
Recording: If no file exists, the user records and saves a new MP3 via the web interface.
Trigger: Arduino sends a signal via serial port when weight exceeds the threshold.
Analysis: Local Vision AI (via LM Studio) analyzes the object upon receiving the signal.
Action: Plays the corresponding MP3 from local storage if the object is recognized.
Recording: If no file exists, the user records and saves a new MP3 via the web interface.
We built a custom web platform utilizing a local Large Vision Model to categorize objects and store voice data. Seeing the system successfully process a bundle of keys for the first time was a truly magical milestone for our team.
We built a custom web platform utilizing a local Large Vision Model to categorize objects and store voice data. Seeing the system successfully process a bundle of keys for the first time was a truly magical milestone for our team.
We built a custom web platform utilizing a local Large Vision Model to categorize objects and store voice data. Seeing the system successfully process a bundle of keys for the first time was a truly magical milestone for our team.
Integrated physical buttons for recording and playback into a minimalist form factor. We also designed status patterns using sound and LEDs to indicate AI analysis, recording, and playback states.
Integrated physical buttons for recording and playback into a minimalist form factor. We also designed status patterns using sound and LEDs to indicate AI analysis, recording, and playback states.
Integrated physical buttons for recording and playback into a minimalist form factor. We also designed status patterns using sound and LEDs to indicate AI analysis, recording, and playback states.
Defined specific sound and light patterns for Analysis, Recording, and Playback modes
Defined specific sound and light patterns for Analysis, Recording, and Playback modes
Defined specific sound and light patterns for Analysis, Recording, and Playback modes
Analyzing Objects
Analyzing Objects
Analyzing Objects
Initial Trigger: LED activates upon object placement
Initial Trigger: LED activates upon object placement
Initial Trigger: LED activates upon object placement
Processing Feedback: Light animations and loading audio signal the start of AI analysis
Processing Feedback: Light animations and loading audio signal the start of AI analysis
Processing Feedback: Light animations and loading audio signal the start of AI analysis
Recording Audio
Recording Audio
Recording Audio
Missing Data: Red signals the absence of a stored audio file for objects
Missing Data: Red signals the absence of a stored audio file for objects
Missing Data: Red signals the absence of a stored audio file for objects
Voice Recording: Long-press the right button for recording, with an auto-save feature upon release
Voice Recording: Long-press the right button for recording, with an auto-save feature upon release
Voice Recording: Long-press the right button for recording, with an auto-save feature upon release
Playback Audio
Playback Audio
Playback Audio
Playback: A rainbow signals successful recording and plays new audio
Playback: A rainbow signals successful recording and plays new audio
Playback: A rainbow signals successful recording and plays new audio
Automated Playback: If a stored memory is recognized, the system immediately triggers the rainbow light and audio playback.
Automated Playback: If a stored memory is recognized, the system immediately triggers the rainbow light and audio playback.
Automated Playback: If a stored memory is recognized, the system immediately triggers the rainbow light and audio playback.
Idle state: Lights turn off upon object removal, returning to idle state.
Idle state: Lights turn off upon object removal, returning to idle state.
Idle state: Lights turn off upon object removal, returning to idle state.
The housing was designed via 3D modeling for precision, then fabricated by laser-cutting and heat-bending acrylic sheets for a snap-fit assembly.
The housing was designed via 3D modeling for precision, then fabricated by laser-cutting and heat-bending acrylic sheets for a snap-fit assembly.
The housing was designed via 3D modeling for precision, then fabricated by laser-cutting and heat-bending acrylic sheets for a snap-fit assembly.
< Components & Assembly >
< Components & Assembly >
< Components & Assembly >
We hosted a lively demonstration, with many participants enjoying the experience of preserving memories with their own bags' contents or our curated objects.
The most rewarding moment of the exhibition was seeing a mother and child record a song for their favorite doll. Watching their joy as the memory played back through my device was a profound reminder of why I create.
We hosted a lively demonstration, with many participants enjoying the experience of preserving memories with their own bags' contents or our curated objects.
The most rewarding moment of the exhibition was seeing a mother and child record a song for their favorite doll. Watching their joy as the memory played back through my device was a profound reminder of why I create.
We hosted a lively demonstration, with many participants enjoying the experience of preserving memories with their own bags' contents or our curated objects.
The most rewarding moment of the exhibition was seeing a mother and child record a song for their favorite doll. Watching their joy as the memory played back through my device was a profound reminder of why I create.