Hello From Yves to the World
To whom it may concern, following are a few projects I have either led or worked on. I hope this will help you get a sense of my past experiences and strengths. Just a disclaimer, these projects are mostly confidential. We can talk about them in detail in person.
Lightning to USB Camera Adapter
The project is based on STM32L011F4. It uses a TIMER peripheral to simulate a SDQ which is an Apple Lightning protocol over ACC_ID Pin. It makes iDevice treat it as an official Lightning to USB Camera Adapter connector. It communicates with a pair of Lightning Data differential lanes using UART peripheral, this process contains certificate exchange and random number challenge / response. The other pair of Lightning Data differential lanes are used by iDevice as USB Host D+/D-. Now, user can treat iDevice as a standard USB host which supports UAC / NCM / ECM / RNDIS / MSC / HID etc. (depends on O/S support). iDevice's Lightning port current may be limited to 100mA in this method. Figure 1 shows how the connection was made from Lightning to USB Host Adapter PLUS. (attached a USB hub after D+/D-, and it can reverse charge iDevice by inserting a Lightning charging cable with the help of receptacle handling on STM32L011F4)
Figure 1. Lightning to USB Host Adapter PLUS
A Normal MFi Involved UAC Device
The project is based on STM32F401CE. According to Apple iAP2 protocol, It communicates with iDevice through the C68A Lightning connector. With the help of CP 2.0C chip, it can process certificate exchange and random number challenge / response. Then iDevice treat the accessory which implemented UAC as a standard USB sound card. Figure 2 shows Lolly 3D Lightning Microphone, which was MFi certified.
Figure 2. Lolly 3D Lightning Microphone
There is also an auto stereo rotation function in Lolly, no matter which direction the connecting iDevice is held, it just synchronizes with the iDevice screen rotation. Figure 3 shows the microphones arrangement. Meanwhile, a reverb effect is included in Lolly. It is a very low latency between USB and I2S, thanks to the elegant synchonization mechanism of USB ISO transfers in async mode.
Figure 3. Microphones Arrangement
Lightning Digital AV Adapter
Thanks to Daniel https://github.com/danielpaulus/quicktime_video_hack, the transactions between Lightning Digital AV Adapter(a.k.a Haywire) and iDevice are parsed out. iDevice recognizes Haywire by USB VID / PID & interface class / subclass / protocl, and downloads firmware into it (there is no NVM in Haywire, just run in RAM, low cost and efficient, as well as flexible). However, reverse engineering can begin from functional mode, in which Raspberry Pi Zero is enumerated like Haywire. It is obvious the H.264 video stream is put on the screen next.
Photos of the implementation on Raspberry Pi Zero are missing. But I can remember the happiness when the "play icon" appeared at the top-right corner of iPhone screen.
CarPlay on Raspberry Pi Zero
It is a little difficult to reverse engineer CarPlay. Even for a normal MFi developer, Apple does not disclose any details of CarPlay on specs. In addition, all the transactions after authentication are encrypted. Luckily, with the help of Lightning & USB protocol analyzer, the transactions between my Mercedes and iPhone were dumped. Plus the previous spec from Apple, which contains the older version CarPlay details, the project was launched. Raspberry Pi Zero uses a pair of IN / OUT endpoints to finish the authentication like an MFi USB NCM device. Then iPhone treats it as a standard ethernet card, so transactions takes place on TCP/IP. With the help of avahi (mDNS service on Linux), iPhone gets the IPv6 address and CarPlay service port. An AES master key is generated after another authentication on network. Different channels (control / audio / screen) with own AES key / IV are setup then. It is RTP protocol for screen channel, and the H.264 Carplay video is put on the screen. Figure 4 shows a part of ATS CarPlay session transactions. (For the reverse engineering of SideCar protocol, USB transactions between iPad and iMac were dumped. Those are also USB NCM transactions. The .csv dump file from totalphase USB analyzer was converted to the .pcap file, which can be parsed by Wireshark. However, there is not enough information to crack this protocol. But it is still under analyzing, Figure 5 shows a part of .pcap transactions)
Figure 4. CarPlay Session
Figure 5. SideCar Session
USB PD to MagSafe 2
Just a background of why I did this project. I bought a new MBP in 2018. Comparing to my previous MBP, it stopped using MagSafe 2 and changed to USB-C instead. I still used my previous MBP every now and then. One day, the MagSafe adapter broke. Since output voltage of MagSafe 2 charger is 20 Volt, so as one of the USB PD port, rather than buying a new charger, I built an USB-C to MagSafe converter so that I can reuse the charger of my new laptop. (Similar as what I did, Apple recently released a cable that connects USB-C to MagSafe port. https://www.apple.com/shop/product/MLYV3AM/A/usb-c-to-MagSafe-3-cable-2-m?fnode=ab6bac0e67597a206dd60fa155b128fcbbfd9449ffdbed1566069b274fefe1f4682f42097b411b9d52497dcfc53a609431a7f429058580343117c25d0c0a5358f83e743c9e794c4b2d0bfc306b963572ebd3b7be8da766713d9a7785faca36c1) In the project, PD physical layer is built by only discrete components, including MOSFETs, resistors and capacitors. Protocol stack is built on STM32L011F4. It uses a TIMER peripheral to receive and send Biphase Marking Code over USB-C CC pin. The whole framework contains physical layer, protocol, policy engine and a local device policy manager. Here is the .c file which has core functions http://jinzhi.xyz/usbpd.c. At the same time, thanks to Ken's blog http://www.righto.com/2013/06/teardown-and-exploration-of-MagSafe.html, it becomes possible that charging MBP through the existent MagSafe 2 port in broken adapter. Figure 6 shows how the connection was made from USB-C power adapter to the old MBP's MagSafe 2 port, and Figure 7 shows the schematic.
Figure 6. USB PD to MagSafe 2 Adapter
Figure 7. Schematic of USB PD to MagSafe 2 Cable
MagSafe 3 Charger
In the project, Qi physical layer is also built by lowest integrity components, which are power MOSFETs, resistors, capacitors, and coil. Qi 1.2.3 protocol is implemented on STM32L011F4. It is the hardest part that demodulating signal from receiver to transmitter. Resonant signal is undersampled by an ADC peripheral at a precise cycle, that means a digital downconversion is made from modulated signal to the baseband one. A valid packet on bus or not is checked through eye diagrams. If so, the threshold between 0 and 1 is determined. After that all the works are in digital domain, it's not too hard to do next. When Apple released the MagSafe 3 wireless charger in 2020, the project was adapted for the new protocol. Although it is not perfect, it can only activate the charging animation. According to the reverse engineering, there may be something about out-of-band communication, such as NFC. But until now, it is hard to imagine the project does work on such a resource-constrained MCU. When putting it on a standard Qi charger, the device becomes a Qi receiver. Figure 8 shows the device, and Figure 9 shows the charging demo, while Figure 10 shows the schematic.
Figure 8. MagSafe 3 Charger
Figure 9. MagSafe 3 Charging
Figure 10. Schematic of MagSafe 3 Charger
Karaoke Reverb System
The block diagram is shown on Figure 11 and Figure 12. There are howling suppression, reverb, and frequency domain filter functions. After simulation on Matlab, the algorithm are implemented on an ARM Cortex-A7 with NEON CPU. Here is the core codes http://jinzhi.xyz/reverb.c.
Figure 11. Reverb System Block Diagram
Figure 12. Howling Suppression Block Diagram
Offline MCU Programmer and Online Signer
After the success of the project MFi UAC Device, the SDK was licensed to a few companies. And it is necessary to deliver an offline programmer and an online signing service. In the project, the JTAG / SWD & USB DFU host are implemented on STM32L475VG. It has ICP and ISP functions. Here is the user instruction which was finished that time http://jinzhi.xyz/instruction.pdf. Signing service is based on PKI - RSA 2048, to be precise. It uses OpenSSL library and is deployed on a Debian VPS, where this page is located. SHA256 hash of MCU's UDID is signed through the service. While on the firmware side of MCU, the signature is checked to assure matching the UDID, by using mbedtls. Figure 13 shows a part of signing log.
Figure 13. Signing Log
USBMUXD on STM32
There is a famous project named libimobiledevice on Github https://github.com/libimobiledevice/libimobiledevice. It must be very cool if libimobiledevice can be implemented on STM32. The project achieves the whole process of simulate location function. It is based on STM32F401CE, which has a full speed USB OTG controller. As a USB host, STM32F401CE recognizes the attachment of an iPhone, and it connects iPhone's lockdown service whose port number is 0xf27e using TCP through USBMUXD. After handshaking by exchanging certificates, an SSL / TLS session is setup. Then a service named com.apple.mobile.mobile_image_mounter is opened through lockdown. It is used for uploading corresponding image and signature which could be found in Xcode to iPhone. Then the com.apple.dt.simulatelocation service is opened. By connecting that service and sending latitude and longitude to it, the specific location is simulated on iPhone, so is the App of Maps. There is no batteries or external power supplies on the device. It can be powered by iPhone directly through Lightning port. This inspiration comes from C68B Lightning connector. Figure 14 shows the Simulate Location Lightning Dongle and below is a demo video.
Figure 14. USBMUXD on STM32
Not all the interesting projects are listed here. I have been persistently exploring the unknown, especially those things are really cool.