Guitar tuner that uses phone accelerometer

(tautme.github.io)

169 points | by adm4 4 days ago

12 comments

  • Tade0 1 day ago
    My accelerometer apparently reads at 200Hz, but due to a lack of instrument at the ready I had to "pluck" the handle of the office fußball table.

    When the right defender is near the center I'm reading ~24.74Hz, so slightly above G.

    • tclancy 1 day ago
      This was the direct inspiration for Total Football: to find harmony with the pitch.
    • virgil_disgr4ce 1 day ago
      heh, a subacoustic G#?
      • Tade0 1 day ago
        I think the tuner is off by an octave - it has a lot of harmonics and sounded like a G on a bass guitar.
        • shiandow 1 day ago
          It seems odd that it would be an octave too low. If it detected any harmonics it would be too high.

          Unless it didn't function like a string at all, then the harmonics would be all wrong.

          • cjbprime 7 hours ago
            > If it detected any harmonics it would be too high.

            I think it's not that simple. A tuner is "hearing" the fundamental and all of the harmonic overtones combined. It has to guess at which frequency is the fundamental, even if the overtones are actually stronger than it amplitude-wise, and it does that by looking at the nature of the repeating overtone pattern and extrapolating back to the fundamental.

            I think you can end up an octave too low (half the actual frequency) if the waveform repeats in a way that implies a different overtone repetition pattern, for example if there's an every-other-cycle artifact to the waveform.

  • Akuehne 1 day ago
    This has some very interesting privacy and security risks. If the tech can do more complex frequency analysis, then couldn't it essentially be used as a microphone for a device that doesn't need permission.
    • smallnix 1 day ago
      I thought this has been done to capture keystrokes of a keyboard next to the phone already

      2011 https://www.researchgate.net/publication/221609349_spiPhone_...

    • pc86 1 day ago
      It's a pretty well-known exploit that the CIA is capable of turning a lot of electronics with speakers into microphones. I imagine there is an entire classified backlog of things they can turn into microphones without the target's knowledge.
      • ge96 1 day ago
        Tangent but the hidden no-electronics bug "The Thing / Great Seal Bug" really crazy
        • maxbond 1 day ago
          It had no internal power supply, it worked like an RFID tag, but it was electronic.

          Tangentially, I didn't know this (from Wikipedia):

          > The Thing was designed by Soviet Russian inventor Leon Theremin, best known for his invention of the theremin, an electronic musical instrument.

          • ge96 1 day ago
            Yeah I was going for no battery

            Idk though when I read it, it seems like it's literally an antenna attached to a can "resonator" is that electronics? It is I guess since it can carry an RF wave? Electronics I think of a chip or circuitry. I get it has to be some form of a circuit to work even as a monopole.

            The article says it though: "...hung in his office behind his desk, and which contained an electronic device"

            • maxbond 1 day ago
              I think if you saw it on a bench hooked up to wires, it would be intuitive that it was a circuit. It's equivalent, but instead of being coupled via wires it's coupled via RF. I think it feels like there's no return path and that the circuit is open, but it's a real circuit with complicated/uncommon coupling to the power source.

              A resonator is both a component in the circuit (the case is a cavity resonator) and the type of circuit this is. When illuminated (or hooked up to a power supply on the bench), it produces a sine wave, and holding all else equal the frequency is a function of the capacitance of our membrane capacitor. That membrane is flapping about due to sound, changing the distance between the plates of our capacitor and thus it's capacitance. So this shifts the frequency we're resonating at and encodes the audio into our output signal (frequency modulation).

              So it's very similar to a standard LC resonator circuit you might make on a breadboard.

              I'll leave you with another story of clever KGB sabotage. The KGB controlled facilities used to construct the US embassy in Moscow in 1979. They were able to extensively bug the building. They were also able to mix thousands of diodes into the concrete. This defeated NLJD (https://en.wikipedia.org/wiki/Nonlinear_junction_detector) based bug detection because they detected the diodes in every direction.

              • ge96 1 day ago
                Thanks for the explanation and will look into NLJD, really need pictures, that bee is cool
      • MomsAVoxell 1 day ago
        The accelerometers that protect the average hard drive are easily subverted for this purpose.
        • sandworm101 1 day ago
          There is something better. The little sensor that maintains the distance between the spinning platter and the armature is sensitive enough to be a reasonable microphone. But it is inside a heavy metal box (the HDD) so you do need to shout at it.

          https://physics.aps.org/articles/v12/24

          >> They tapped into the feedback system that helps control the position of the read head above the magnetic disk. When the head is buffeted by sound waves, the vibrations are reflected in the voltage signal produced by the drive’s position sensors. By reading this signal, Fu and his colleagues were able to make high-quality recordings of people speaking near the drive.

          • fennecbutt 1 day ago
            Good old video of a guy shouting in a data center https://youtu.be/tDacjrSCeq4?si=ebFDFYufOdNIU9av
          • ok_dad 1 day ago
            The NSA could turn on your flip phones mic thirty years ago without you knowing, I don’t think they needed to do all that fancy stuff with hard drives. That’s just research that they funded to cover up the fact that they owned every computing device on the planet for a while.
      • amelius 1 day ago
        I mean at this point I'm going to assume that any semiconductor device with more than a few pins has an embedded mems microphone.
      • righthand 1 day ago
        The CIA…plug a set of regular headphones into a microphone jack, open a recording application and speak into the headphone speaker, you don’t need a 3 letter agency for that physics open secret.
        • nerdsniper 1 day ago
          Wouldn't you need to rewire the headphones? Headphones use a 3-pin TRS whereas a 4-pin TRRS plug is used when you add a microphone. Regardless if the 4-pin is CTIA or OMTP, it's generally only going to get shorted to ground if a 3-pin TRS plug is plugged into a 4-pin TRRS socket, or if a 4-pin TRRS plug is plugged into a 3-pin TRS socket.

          Diagram: https://i.sstatic.net/8rSD2.jpg

          • kps 1 day ago
            Non-phone non-Apple devices often have a TRS microphone input separate from the TRS headphone output.
          • lightedman 1 day ago
            "Wouldn't you need to rewire the headphones?"

            This is basic physics controlling the effect here, not electrical routing. Speakers are microphones by their very design. To make them work as a microphone, you merely speak into them with them plugged into an input jack that provides at minimum a line level electrical signal to be modified by wiggling the speaker cone/diaphragm back and forth.

            • nerdsniper 9 hours ago
              Yes, but the computer doesn't have the firmware to "record" that signal from the speaker output pins. Thus, to record from the speakers acting like microphones, would require rewiring the headphone cable, for the vast majority of computing devices.

              If you click "record" on your computer, there's no way to tell it to record signal from the speaker output channels, even if you write a custom low-level application directly making OS calls. The OS can't even do it, because it's not supported by the firmware.

        • tclancy 1 day ago
          I am crap with physics but was going to say I think the last 50+ years of speaker development has been about making them less a microphone than they inherently are.
          • ssl-3 1 day ago
            No, not really.

            Dynamic loudspeakers and dynamic microphones are the same thing. They always have been the same.

            They've got the knobs for the design variables turned in different directions, but they're still the same.

            They even have the same frequency response whether they're being used as speakers or microphones at the moment.

            Which brings up a valid way to measure the response of a microphone's design:

            Use two of them. One as a speaker, and the other as a microphone. Play measurement-sounds out of one, and record the results on the other. Plot it out.

            The deviations are magnified, but eliminating that magnification is just a math problem -- not an instrumentation problem. :)

          • righthand 1 day ago
            They transmit sound. Anything able to detect the vibrations make it a microphone. Not sure how a speaker gets around that because it’s job is to vibrate.
      • _blk 1 day ago
        it's not (just) the CIA, it's (just) physics
    • jrflo 1 day ago
      I don't think that's realistic. If you're looking at the acceleration sound waves cause against a phone's accelerometer, that's likely far below the sensitivity of the sensor- phones are too massive relative to the force of sound waves from speaking. F=ma, so the acceleration you're looking at is the force of the soundwave (tiny) divided by the phone's mass (relatively large). The only reason this kind of works is because you're putting the phone on an object that's mechanically vibrating. I suppose it would work in certain situations like putting the phone on top of a large speaker, but you'd never get the resolution to decipher audio from sound waves alone for a phone sitting on a desk or in a pocket
    • jjk166 1 day ago
      I doubt the sampling rate is anywhere near what you would need to make out dialog in a sound recording. You might be able to tell who is speaking though if you had a voice profile.
    • weard_beard 1 day ago
      Sounds like you've got a great idea for a proof of concept for DefCon next year...
  • dehrmann 1 day ago
    The more reliable guitar tuners do something like this. You clip them on the neck, and they detect vibrations in the wood rather than from sound in the air.
    • foobarian 1 day ago
      Violin/other strings tuners tend to have this mode for orchestra uses with many people next to you also making sounds.
    • soperj 1 day ago
      oddly, this is how I learned to tune my guitar. I'm pretty tone deaf, but can feel the difference between the reference string and the string I'm trying to tune through the guitar.
      • DANmode 1 day ago
        This is fascinating.

        Thanks for sharing.

        • adzm 1 day ago
          The beat frequencies are pretty noticable, just tune until they go away
          • soperj 1 day ago
            That's the terminology I was looking for. Didn't know whether to tune up or down without changing frets, could tell when it was right though.
          • hunter2_ 1 day ago
            Not really. 12 TET (the tuning intervals used in almost all situations where you want equal ability to play in any key, from a Western music perspective) requires a slight amount of beating, even in those "perfect fourth" intervals between most adjacent guitar strings, and especially in the "major third" interval between one pair of adjacent guitar strings, but not in the "2 octave" interval between the lowest and highest guitar strings.

            Some equal temperament intervals are narrower than their just-intonation (nonbeating) counterparts, for example an ET perfect fifth (2.996614:2) and just perfect fifth (3:2). But others are wider, for example an ET perfect fourth (4.00452:3) and just perfect fourth (4:3), and an ET major third (5.039684:4) and just major third (5:4).

            If you tune each string sequentially (low E to high E, or vice versa) and eliminate all of the beating each step of the way, the effect adds up to the point where it's quite noticeable, meaning that your low E and high E will sound like garbage when played together because that ratio should be precisely 4:1 but now you've accidentally made it narrower than that. How much narrower?

            For a guitar going from low E string to high E string, we need to stack 3 perfect fourths, a major third, and another perfect fourth -- and end up at 4x the starting frequency. If we use those non-integer ratios of the 12 TET system (intentional beating), we end up with 4.00452/3 * 4.00452/3 * 4.00452/3 * 5.039684/4 * 4.00452/3 = 4.00000. That's what we want. But if we use the integer ratios of just-intonation (no beating), we end up with 4/3 * 4/3 * 4/3 * 5/4 * 4/3 = 3.95062 and that's going to sound like complete ass. This is why just-intonation is not used for instruments like piano and guitar that are designed to play in all keys. It's used by choirs and barbarshop quartets without piano accompaniment, since they can adapt on the fly, and it's glorious.

            An electronic tuner is the most practical way to avoid this problem. Alternatively, you could just get the perfect fourths nonbeating, get the double-octave nonbeating, and let the major third beat however it wants -- this works because rounding the ~4.005:3 perfect fourth to 4/3 is somewhat acceptable but rounding the ~5.04/4 major third to 5/3 is not.

            • chupasaurus 23 hours ago
              Perfect fourth could be achieved via natural harmonics at 5th and 7th frets, double octave is just the 5th fret and voila you can get the standard tuning. Inapplicable for necks without a precise position of frets, good luck compensating height differences from nut and gauges with intonation tuners.
              • hunter2_ 19 hours ago
                Oh yeah, I should've mentioned fretting the lower strings for a unison (1:1) to the higher adjacent open string. No beating in this unison yields equal temperament, due to frets scaled accordingly, so it avoids the problem I mentioned... but introduces likelihood of intonation issues (or simply bending the pitch) as you say.

                I had been thinking about listening for beating in the 5:4 and 4:3 intervals using open strings, which does present the just temperament problem, and harmonics on open strings to make a unison unfortunately shares this problem 100%.

  • jmusall 1 day ago
    Fun idea and also I didn't know that websites could get access to my accelerometer data. However for me the sample frequency is 50 Hz which is way too low to measure even the lowest string pitch (E2, about 82 Hz).
    • hgomersall 1 day ago
      If you know you have a single frequency close to an actual frequency of interest, you can use the fact you know you're in an aliased band to get a precise frequency estimate.
      • superxpro12 1 day ago
        I guess thats sort of like a weird PLL thing? But I'd imagine you'd have to have prior knowledge of which string you're tuning otherwise the analysis is going to alias against every harmonic.
      • jonathrg 1 day ago
        Presumably there is an antialiasing low pass filter somewhere before JS gets to the data. I have a similar sample rate and it certainly didn't work at all for me.
        • regularfry 1 day ago
          They have analogue AA filters just before the sampler.
        • ErroneousBosh 1 day ago
          If the accelerometer samples at 50Hz, how could there be an antialiasing filter?

          What would that filter look like?

          • colanderman 1 day ago
            Anything physical which dampens higher frequency oscillations would act as an antialiasing filter.
            • ErroneousBosh 1 day ago
              What sort of size do you think something that would damp 25Hz vibrations in something that weighs a gram or two would need to be?
      • ckocagil 1 day ago
        aka a stroboscopic measurement,

        but I don't think it will work well for this case.

        • KeplerBoy 1 day ago
          It's just higher nyquist zones.
    • peheje 1 day ago
    • lightedman 1 day ago
      "the lowest string pitch (E2, about 82 Hz)."

      My 6-string Kiesel Kyber bass would like a word with you while it sounds 41Hz.

      • ta2112 1 day ago
        I guess the low B should be about 31Hz
  • adm4 4 days ago
    guitar detuner that uses accelerometer instead of microphone, it doesn't really work, but amazing to see how sensitive they are.
  • donclark 1 day ago
    Can an accelerometer determine when a car is having issues? Can it do the same for a human user's body?
    • adm4 22 hours ago
      i remember hearing Porsche was doing some machine learning with vibration analysis to find car trouble. https://www.porsche.digital/en
      • emil-lp 16 hours ago
        Yes, predictive maintenance on motors and turbines using vibration is quite common.
  • kingkawn 1 day ago
    The very clear and succinct description on the landing page makes me miss the bizarre antisocial charming quirk that people who made things like this used to be stuck with for their copy rather than AI generated language. Our cacophony of experience is quieting.
  • JoheyDev888 1 day ago
    The neat bit is that it doesn't necessarily need to sample 82 Hz directly. If the sample rate is known and the target is one of a few guitar strings, the aliased peak can still be useful. The tricky part is probably rejecting the wrong alias once the vibration signal gets messy.
  • ramenat2am 1 day ago
    I mean yeah, that's cool as a fun project. And I've also heard about a project that used accelerometers as microphones for surveillance. And while it's doable, even the cheapest crappiest mic would do a much better job at recording sounds for whatever is your goal.
    • embedding-shape 1 day ago
      > even the cheapest crappiest mic would do a much better job at recording sounds

      And if you don't even have that, use a speaker/headphone as the microphone, probably also better results.

      • kelipso 1 day ago
        It’s about what apps can do with just default permissions, no? Not about what’s theoretically possible given full access to the phone.
  • AtNightWeCode 1 day ago
    Cool. It worked for E and A but it failed for string 1-4. I was surprised that it worked at all.
  • aa-jv 1 day ago
    Anyone got a handle on the algorithm required to do this? I've got a pocketable accelerometer-enabled device I'd like to try to implement this on..
    • simonklitj 1 day ago
      Don't have a handle on it, unfortunately. But the algorithm is in here: https://github.com/tautme/phone-sensors/blob/main/guitar-tun.... Esp. lines 221–257 and 373–417.
      • codedokode 1 day ago
        The code mentions "autocorrelation" method: this is a method where you multiply the signal with delayed version of itself: result = sum(x[i] * x[i - delay] for i in some range). You vary the "delay" and pick the value that maximizes the result. This is based on the idea, that the sequential periods of the signal should be similar to each other.

        Not a very good method, prone to octave errors (showing pitch one octave lower than the correct one). Furthermore, the "delay" is an integer which limits the precision, so you need to use some form of interpolation. Also it doesn't allow to recognize multiple notes sounding together. Also, slow.

        You can read the paper on the "YIN" pitch estimation algorithm which describes the method in details.

        I think FFT-based methods are more reliable. I did little experimentation and when measuring a pure sine wave, the frequency can be determined with high precision (tenths-hundredths of a Herz). Not so good in presence of a noise or multiple instruments - I tried to use descending from the hill optimization to figure out the pitch of each harmonic, but it didn't work out.

        • dsego 1 day ago
          I implemented the McLeod NSDF pitch method, which normalizes the autocorrelation to get a pretty reliable estimate with fewer octave jumps. For precise tuning I used phase tracking between successive single-bin DFTs tuned to the target frequency.

          https://github.com/dsego/strobe-tuner/

      • aa-jv 1 day ago
        Ah, that does look like something I can work with - thanks for the legwork, I will check it out and see if its worthwhile converting to C/C++ for my device ..
  • nubinetwork 1 day ago
    This sounds neat, but I think I owned a tuner for about 6 weeks before I could do it by ear... EADGBe isn't that hard.
    • jameshart 1 day ago
      Pack it up folks, nubinetwork has exposed the scam that is the guitar tuner industry. You don’t need a guitar tuner if you have ears; all the guitar techs and musicians who use them have bought into a lie. And obviously since guitar tuners are a waste of time, a tech demo showing that you can use the accelerometer in a commodity handheld device to pick up minute vibrations with sufficient accuracy to detect guitar tuning from a web page is just feeding into the hands of Big Tuner.

      Seriously, this is the very definition of a shallow dismissal.

    • beezlebroxxxxxx 1 day ago
      If you have a good external reference point. But it's also pretty easy to have your tuning drift quite a bit away from E standard if you solely rely on the strings. Getting a standard tuning is not the same as getting the standard tuning you want, exactly. This is especially true if you play in standard tunings below E, like C or B, where strings can be looser than the norm.
    • markvdb 1 day ago
      Pro guitar teacher here with over twenty years of experience teaching the guitar, and close to fourty years of experience playing the guitar. I still struggle with properly tuning my instrument by ear. Nothing wrong with my ears. It's just not easy to do this right.
      • dsego 1 day ago
        For me it's hard because of tempered tuning, so each string should be slightly out of tune for everything to sound good. If I tuned by ear, I could get two open strings to match, but all the fretted notes and chords wouldn't sound good. On my ukulele I even tune one string down on purpose to make the fretted note sound better. And then there is the inharmonicity of overtones, and some strings have more noticeable overtones that influence how the pitch is perceived.
    • recursive 1 day ago
      So what do you do when the lead singer is engaging in some stage banter and you need to tune between songs? IEMs? With what reference pitch?
      • mrguyorama 4 hours ago
        >IEMs

        Absolutely. In orchestral band back in my high school days, between songs I would ask my friend on the tuba to give me a note for reference so I could tune the timpani. If you are in a band you should be able to manage to take a one second note and hold that by humming or something and tune up. I can't even read real music (I was a "percussionist" ie incompetent except at rhythms) and even I could do that. Better trained musicians than I could expand from there to the other notes you need.

        You just need a single reference pitch. You can literally just play a sine wave into your headphones for a second on stage, or have your audio engineering guy feed you one. Or you have one of those dirt cheap tuners that clips to the fretboard to get you started.

        The lead singer is engaging in stage banter partially to give you the time and space to do this. If your ensemble includes a pianist or a synth, you just have them slap a note for a reference.

      • lightedman 1 day ago
        A good chunk of us are in tune (pun intended) with our instruments to the point that we can simply feel the vibrations and know where we are with regards to tuning.

        I haven't touched a tuner in about half a decade.

        • recursive 1 day ago
          That sounds like a very useful skill for a working player. But it seems to come with a couple of significant conditions.

          I can believe this is possible. But I don't think this is a reasonable thing as a baseline expectation for a player with 6 weeks of experience, which was the original comment in this thread.

          I don't know the details, but I imagine you're feeling beats transmitted through the neck or something. But if that is the case (an assumption) it still requires you to have at least one known-good string, unless you're playing solo.

          So, for these circumstances, and others, a pedal-based or clip-on headstock tuner seem like they still have plenty of practical application.

    • chpatrick 1 day ago
      It's not hard if E is in tune already. :)
    • Hamuko 1 day ago
      I can tell if my guitar's significantly out of tune, but no way I'm getting an accurate tune without a tuner.
    • GuinansEyebrows 1 day ago
      i'm like this at home for sure, until i want to play along to something in tune.

      i'm also the one at rehearsal literally throwing TU-3s at my bandmates who don't have tuners on their boards for some reason. you have to have a tuner if you play with others. no question.