• This topic is empty.
Viewing 15 posts - 1 through 15 (of 18 total)
  • Author
  • #10058

    Stage Complete
    Cost to Build $250CAD
    Project Need Mobility, Other
    Created By Makers Making Change
    Capabilities Needed 3D Printing, Electronics, Software, Soldering
    Time to Complete 6 print hours + 2 to 3 assembly
    Creative Commons License Attribution-NonCommercial-ShareAlike

    The Lipsync Wireless is a mouth operated joystick that allows a person to wirelessly control a computer cursor with minimal head and neck movement. All the electronics are housed in the ‘head’ of the device so there are no additional control boxes, making the LipSync a good candidate for portable, wheelchair-mounted applications. The mouthpiece is attached to a precision miniature joystick sensor that requires very slight pressure in order to move a cursor on the screen.

    Resources GitHub (All files, manuals, 3D printed parts, and BOM)


    **LipSync Wireless**
    [LipSync Wireless User Guide](https://github.com/makersmakingchange/LipSync-Wireless/blob/master/LipSync_Wireless_User_Guide.pdf)
    [LipSync Wireless Startup Guide](https://github.com/makersmakingchange/LipSync-Wireless/blob/master/LipSync_Wireless_Startup_Guide.pdf)

    **We’re looking for feedback on the LipSync Wireless** . Your assessment is critical to making sure we design relevant technology for end users and that it has the features and performance they need. Please take a few minutes to give us your thoughts.

    * What was your experience using the device?
    * What worked well for you/your client?
    * What was the nature of disability and severity of the specific type of user that you tested the device with?
    * What would you like to see in the next design changes?
    * Was it responsive enough?
    * Was it easy to setup?
    * Are the instructions on how to use it obvious and easy to understand?
    * What was missing from the design?



    My name is Elizabeth, personal caregiver, and I am replying with feedback on behalf of my client.

    My client has advanced relapsing-remitting multiple sclerosis. Currently he is wheelchair-bound, has no use of his legs or left arm, limited use of his right arm, but full neck mobility. His speech is declining, and the primary intent behind use of the LipSync for him was to control his wheelchair-mounted tablet computer and speech app.

    The LipSync itself was very easy to set up and responsive for him. It was more intuitive and easy to use than some other hands-free computer systems that he had tried, and he was pleased that the replacement cost was low in case of anything happening to it. (He lives in a nursing facility and has unfortunately had assistive equipment broken by staff in the process of personal care, which the facility operators generally do not pay to replace.)

    However, one major obstacle to functionality has been the issue of mounting. The Neil Squire Foundation informed us that they are working on wheelchair mounting solutions for the LipSync, but what they had to offer initially was too heavy and unwieldy to mount to his chair for longer than short periods of practice. He was eventually able to purchase a lighter, flexible mount for himself, but the wheelchair technician mounted it out of his reach and was not able to come back and correct the problem quickly. Most recently the LipSync mounting placement was fixed, but then the mount broke off of his chair :frowning:

    Given all of these obstacles, even with the assistance of a personal caregiver, he has not been able to maximize his use of the LipSync, or even to really meet his goals thus far with the device.


    We had a great build event with chapter leaders last night. Some of them had difficulty programming the LipSync Wireless. Here’s a draft process that should help resolve the issue:

    # Programming non-responsive LipSync Wireless boards

    Use this process if your LipSync Wireless isn’t visible in your Bluetooth device menu. If possible, reprogram your LipSync Wireless with the case removed, or at least while able to see inside the unit from the screw mounting. There is a blue flashing status LED that tells you if the process has worked or not.

    ## Cause

    The Bluetooth board inside the LipSync Wireless needs to be programmed before it will act as a mouse. Sometimes these boards are shipped to you without being programmed. If your Bluetooth board was still in the sealed *SparkFun Bluetooth Mate* antistatic bag, then it has not been programmed. The LipSync Wireless code contains the programming routine, but it is not normally enabled.

    ## Source

    This has been modified from Part 8A of the [LipSync Manual](https://raw.githubusercontent.com/makersmakingchange/LipSync/master/LipSync_Manual.pdf) combined with internal guidance from Makers Making Change engineer Milad Hajihassan. The process has been updated to follow small UI changes in the Arduino 1.8.13 IDE.

    ## Process

    You will need to upload the firmware twice: the first time will program the Bluetooth board; the second time will install the correct LipSync Wireless firmware. Modifying one line of code is required.

    1. Download and install the latest version of Arduino IDE: https://www.arduino.cc/en/software

    2. Download and save the latest [LipSync Wireless microcontroller code]( https://raw.githubusercontent.com/makersmakingchange/LipSync-Wireless/master/Software/LipSync_Wireless_Firmware/LipSync_Wireless_Firmware.ino) . Right-click on the hyperlink of the microcontroller code and select “Save Target/Link As”

    3. Open the Arduino IDE and click on Tools → Board → Arduino AVR Boards → Arduino Micro

    4. Connect the LipSync to the PC using the USB cable and allow the LipSync a few seconds to initialize

    5. Click on Tools → Port → COM?? (Arduino Micro): the port name may have different numbers or name, but the IDE should have recognized it as an Arduino Micro

    6. Open the `LipSync_Wireless_Firmware.ino` you recently downloaded with the Arduino IDE. It may prompt you to accept moving it into a sketch folder, which you should accept

    7. In the firmware code, find the line that starts `#define BT_CONFIG_FLAG false` (at time of writing, [line 52](https://github.com/makersmakingchange/LipSync-Wireless/blob/97e4a3f4462ad395ee632dca42c5ad2c68693162/Software/LipSync_Wireless_Firmware/LipSync_Wireless_Firmware.ino#L52)) and change it to `#define BT_CONFIG_FLAG true`

    8. Save the firmware via File → Save

    9. Click the right arrow at the top left of the Arduino IDE to upload the microcontroller code to the LipSync Wireless device. After a few seconds, the blue LED on the Bluetooth module will start to blink quickly. If you have your Bluetooth host device connection display open, you should see a device called *LipsyncMouse*

    10. Modify the firmware code line to read `#define BT_CONFIG_FLAG false` again

    11. Save the firmware again via File → Save

    12. Upload the microcontroller code to the LipSync Wireless device again by clicking the right arrow at the top left of the Arduino IDE

    13. Once the upload is complete, a message saying “Upload Successful” will show at the bottom left of the Arduino IDE

    14. Perform Joystick Calibration as described in the [LipSync Manual](https://raw.githubusercontent.com/makersmakingchange/LipSync/master/LipSync_Manual.pdf), Part 8B.


    Hi Stewart,

    Thank you for your suggestions!

    As suggested (in email), I disassembled the LipSync so I can see the lights on the boards. Then I followed the Process above (including a fresh copies of the Arduino IDE 1.8.13 dated July 2020 and the LipSync Wireless microcontroller code). For reference, I am trying to use the LipSync wireless on a Windows PC (running 64-bit Windows 10 Version 2004).

    I made changes to LipSync_Wireless_Firmware as in steps 7 – 9 above. With the LypSync connected via USB, under Settings -> Devices -> Bluetooth -> Mouse, keyboard, & pen, the LipSync was listed as Arduino Micro (with mouse icon). Then I followed steps 10 – 13 before reassembling and calibrating the LipSync.

    The LipSync works well as long as it is connected to the PC via USB cable (as a mouse as well as being listed as a Bluetooth mouse). However, when I disconnect the USB cable connecting LipSync to the PC and connected to another power source (cell phone charger), the LipSync was no longer discoverable (automatically nor manually).

    Please advise.





    Hey @alice sorry you are still having troubles.

    The current version of the Wireless Firmware (v1.0) can be used both for USB mode and Wireless mode. When the device is powered on, it looks at whether the shunt connector is installed (J13) to select between the two modes. It sounds like the jumper may still be installed since you are able to use the LipSync as a USB mouse.

    The jumper is installed in Part 1, Step 12. It is removed if making the wireless version in Part 2, Step 10.

    Shunt Connector On: USB Mode

    Shunt Connector Off: Bluetooth Mode

    When a Wireless LipSync is powered (and the shunt connector is removed), it should automatically broadcast the “LipSyncMouse” id over Bluetooth. The red led on the top of the Bluetooth module will slowly flash red. When it pairs, the flashing red light will stop and the green connect LED will light up and remain on.

    We’re planning a number of improvements to the LipSync. Removing the need to upload the firmware twice has already been looked at, and making it easier to switch between USB and wireless is on the list.

    Let us know if this resolves your issues.


    Thanks, @MMC_Jake! I did leave the shunt connector on. (Careful reading of the instructions would have been helpful! 🙁 ) Disconnecting it did the trick!

    Yes, being able to switch between USB and wireless would be really useful.

    THANKS, again!




    I have also personally ran into the same issue – we’ll think about how to make his clearer. (And @MMC_Milad has some good ideas for resolving this through software)


    Hi all! Ha! Mine fell off and I didn’t even notice it. Thankfully I found the little piece. When connecting. Using MacOS, I was able to connect but the connection kept closing (see image)
    ![Screen Shot 2020-11-27 at 5.42.41 PM|690×464, 50%](upload://y0ouhadzk3AThr5aqzCfTjf9xtM.jpeg)

    The device didn’t show up on my iOS bluetooth at all. (I followed the double flashing instructions as well) Not sure if anyone has some leads on why?

    On another note, I think my soldering skills may have gone awry. When just trying to use the lipsync as a usb device I can puff but sip is inconsistent. Also my mouse tends to drift to the lower left once activated. Not sure if that’s a soldering issue or something more mechanical? If anyone has some ideas that would be great before I take it all apart.

    In any case, while trying to troubleshoot, I started making myself a little [web app](https://lipsync-calibration-tester.loretod.repl.co) that distinctly shows the mouse movements and whether a right or left click was detected. I was hoping to add button press event notifications, and such since it might be easier than figuring out what the LED blink colors mean. The serial messages are super helpful which led me into using the [Web Serial API](https://web.dev/serial/) to display and add more granular messages to help me. I am able to connect to the port but get an async error message when trying to open communication. This is beyond my scope of knowledge at this point but I think it would be super helpful on the consumer side to have some insight into what is happening without using the serial monitor in the Arduino IDE.

    In any case, guidance would be most appreciated!


    Hi @loretodalt,

    I’m using the (Bluetooth) wireless LipSync Mouse (LM) on both a Windows PC and MacBook Air running OS 10.14.6. It’s been working on both.

    At first my Mac didn’t see the LM (used as a wireless mouse and plugged into a USB charger) either. So I went back to the System Preferences window and searched for ‘Blue’. Several options appeared and I chose “Bluetooth devices”. Then a window — similar to the one in your message above — appeared, a little circular swirly icon appeared and after a second or two, LipSyncMouse appeared on the list. My puffs did act similiarly to a left-button press (to chose the focus-window). Sips brought up different menus depending on the location of the mouse. (These actions are similar to using an external 3-button mouse on a Mac.)

    Good luck!




    Thanks, @MMC_Jake, but I’m not sure how to make instructions fool-proof … there are too many of us fools around. But if @MMC_Milad can resolve this through software, that’s probably the best anyone can do! BEST OF LUCK, @MMC_Milad!


    @alice Fool-proof is probably too high of a bar – we’re just shooting for fool-resistant 😉


    The ‘RNBT-590A’ is the default ID of the Bluetooth module. (I.e. RN – manufacturer, BT- Bluetooth, 590A is the last 4 characters of the MAC id). This indicates that the Bluetooth module has not been properly configured by the LipSync. When it’s properly configured, it will appear as ‘LipSync Mouse’ in the list of Bluetooth devices (and typically as a hybrid keyboard / mouse) I would recommend trying the dual-flashing procedure again. Make sure to wait a bit after flashing the first time to allow the LipSync to run through it’s routine.

    I like the little web app you created – could be a helpful tool for checking a LipSync post-build.

    We have been doing some initial testing very recently on an app that @MMC_Milad has developed that provides more of a user-accessible serial interface for configuring, troubleshooting, etc. You’re right that the Arduino IDE isn’t very consumer friendly. The changes to the firmware should make it easier.


    Thanks so much @MMC_Jake and @alice. I’m on the right track then. Patience is definitely not my strong suit. I will wait longer between flashes :laughing:

    And I second the thanks to @MMC_Milad for working towards making it more fool- resistant. This fool greatly appreciates it.


    @loretodalt: You need to use the beta version of chrome to get the Serial API working and enable Dev mode. That makes it unpractical at the moment but they will eventually release it to all web browsers.

    I got it working to test sip and puff morse code combination through serial.

    A java applet would be good option as it would be cross platform.

    I have another idea in mind but I have to check if it’s practical first.

Viewing 15 posts - 1 through 15 (of 18 total)
  • You must be logged in to reply to this topic.