Is the first C major correct?

[T-vK]

I'm not an expert, but the first C major chord looks a bit strange to me:

      frets: '332010',
      fingers: '342010'

I usually play it like this:

      frets: 'x32010',
      fingers: '032010'

Your variant looks more like a 'C/G' to me. What do you think?

[tombatossals]

Yess, you're right. It's more standard the Cmajor that you say, I have seen it in multiple sources, some muted the 6th string and others prefers the G note in the 6th chord. Some others represented it as a variation of the same chord.

So you think it would be a C/G variation, very interesting. I'm going to change it like you say. Do you know any source where I could query the "slash" chords so I can add them?

I'm finishing a web visualization tool of all the chords so it would be easier to browse them and detect all the typos that I could make in them.

Thanks

[T-vK]

I couldn't find a good source that includes slash chords and finger positions, unfortunately...

chordsdatabase.com seems to have a pretty nice data base including finger positions, lots of variations for each chord and a lot of suffixes, but no slash chords. I wrote a little script to extract all the data. Maybe this this helps a bit: https://pastebin.com/raw/8kQK8Z37

I saved the data in a simple format like this:

chords: {
    "C#":[{
        "positions": ["x","4","6","6","6","4"],
        "fingerings": [["0","1","3","3","3","1"], ["0","1","2","3","4","1"]]
    },{
        "positions": ["9","11","11","10","9","9"],
        "fingerings":[["1","3","4","2","1","1"]]
    }, ...],
    "C#m":[{
        "positions":["x","4","6","6","5","4"],
        "fingerings":[["0","1","3","4","2","1"]]
    },{
        "positions":["9","11","11","9","9","9"],
        "fingerings":[["1","3","4","1","1","1"]]
    }, ...],
    ...
}

Keep in mind that there is a lot of redundancy in that file because every chord is present in its sharp and flat form (Ab == G#, E# == F, ....).

[szaza]

Hi @T-vK, are you still interested in this topic?
I've just started to add some slash chords, just check this issue: #6;
Would you like to join us for extending the database?

[T-vK]

@szaza Hi, I'm still very much interested. I just haven't had the time to work on it. I did find a pretty complete "database": https://jguitar.com/chordsearch?chordsearch=&labels=finger
I think it truely contains every possible chord including fingering positions. Unfortunately it only shows one fingering possibility per shape, but nevertheless I think it would be worth it using the data.
I'd just have to write a little script that scrapes the data for every possible chord. So basically I need a list of all possible chord names. Then it's just a question of time until all the data is scraped.

Edit:
I think this page would make a good starting point for scraping: https://jguitar.com/chord?root=C&chord=Major&bass=C&labels=finger&gaps=0&fingers=4&notes=sharps

We could then simply have three nested loops. The first one iterating over the root note, the second one iterating over the chord variant and the third one iterating over the bass note. This would result in about 10000 HTTP requests. For some chords we need to send multiple requests as there are so many possible shapes that there are multiple pages.

[szaza]

Hi @T-vK,
I'm very glad that you already started to find a way to obtain all the slash chords, however I'm not sure that fetching the jguitar.com is the best one. My doubts are the followings:

• we would need parseable resources (JSON format would be the best), however as I see jGuitar returns chords only as images (in .png format); Of course we could write an image processing algorithm, that converts images to JSON format, but that would take some time;
• also I do not consider parsing another site as the most ethical solution, maybe it would be better to contact them and request chords from them. As guitar chords are their real value, I don't think so that they will provide us the table of the chords for free, but maybe it worth a try;
• maybe we could find a way to generate all the available slash chords: by transposing base chords we can obtain new chords.

@tombatossals how did you generate the current database? Did you put down all the chords by your hand?

[T-vK]

Processing the images wouldn't be that hard actually and would only be necessary to obtain the finger positions. The rest can be extracted from the URL to each image.
I'm pretty sure they have an algorithm to generate all these chords rather than a database. I don't really see a problem with parsing their public content. I mean if you make something publicly available on the Internet then everybody can use it. I'd obviously give them credits, but since they don't sell that information and I certainly have no plans to do so either, I don't think it would be unethical at all.

[szaza]

Hi,

"Processing the images wouldn't be that hard actually and would only be necessary to obtain the finger positions." - Can we try it out first on a single image? What approach would you use for it? Have you done similar tasks?

"I'm pretty sure they have an algorithm to generate all these chords rather than a database" - yes, I'm also fully sure about it, however it is enough to generate the database only once, because guitar chords won't change by passing the time.

"I mean if you make something publicly available on the Internet then everybody can use it." - Yes, it is free to use, however I'm not sure that it is allowed to make a copy. Just consider using YouTube, it is free to use, but it is not permitted to download the content and put it available on another site.

In my opinion the right approach would be to write an algorithm that generates chords instead of spending time with writing image processing and site parsing algorithms.

[T-vK]

Can we try it out first on a single image? What approach would you use for it? Have you done similar tasks?

Well testing it on one image first would be the first step for sure. I have done similar things. For instance I have written a proof of concept showing how easy it is to decipher a simple captcha. The only difference for the chord diagrams would be that I'd have to find the positions of every number found in the image. https://github.com/T-vK/crack-captcha-poc

In my opinion the right approach would be to write an algorithm that generates chords instead of spending time with writing image processing and site parsing algorithms.

I fully agree, but I know way too little about music theory to do something like that.

Just consider using YouTube, it is free to use, but it is not permitted to download the content and put it available on another site.

You can't really compare the two because you can't have a copyright on a chord. A fair comparison would be if someone uploaded a video that is in the public domain. In that case there are no laws that would disallow making copies of it.

[tombatossals]

@tombatossals how did you generate the current database? Did you put down all the chords by your hand?

Yesss, all by hand. I started it as a proof of concept and finished coding the chords by hand I found in some old books I have.

[szaza]

@tombatossals what do you think about @T-vK's approach?
Can we give a try to obtain the missing chords from jGuitar?

[T-vK]

I haven't had much time to get this to work. But here is what I ended up with on the weekend:

const Tesseract = require('tesseract.js')
const sharp = require('sharp')


async function imageToNumber(filePath) {
    const result = await Tesseract.recognize(filePath, {
        lang: 'eng',
        tessedit_char_whitelist: '0123456789T' // allow letter T and numbers
    })
    console.log(result)
    return parseInt(result.text)
}

const fingerWidth = 15
const fingerHeight = 15
const stringPositions = [18,43,68,93,118,143]
const fretPositions = [55,80,105,130,155]

const baseFretPositionX = 162
const baseFretPositionY = 51
const baseFretPositionWidth = 24
const baseFretPositionHeight = 24

async function main() {
    const inputImg = await sharp('test.png')
    let tmpOutput

    tmpOutput = await inputImg.extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toBuffer()
    //await inputImg.extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toFile('output.png')

    const baseFret = await imageToNumber(tmpOutput)
    console.log('Base fret:', baseFret)

    const fingering = []

    for ([i,string] of Object.entries(stringPositions)) {
        let fingerFound
        let fingerFoundFret
        for ([j,fret] of Object.entries(fretPositions)) {
            tmpOutput = await inputImg.extract({ left: string, top: fret, width: fingerWidth, height: fingerHeight }).toBuffer()
            const finger = await imageToNumber(tmpOutput)
            if (!isNaN(finger)) {
                fingerFound = true
                fingering.push(finger)
                break
            }
        }
        if (!fingerFound) {
            fingering.push(0) // TODO: check if string is muted and push 'x' in that case
        }
    }
    console.log('Fingering detected:', fingering)
    process.exit()
}

main().catch(console.error)

Input:

Output:

Fingering detected: [ 1, 3, 4, 1, 1, 1 ]

So yeah, it basically works, but something is still wrong. It seems to detect a "1" when there is a "2".

[szaza]

It looks pretty good, maybe I'll have some time during this week to refine a little bit your implementation. We also have to calculate the position of the fingers on the frets, so in this case it would be: frets [1,3,3,2,1,1] and also barres: 11 is important.

[T-vK]

Yes, I kinda forgot about that. But the information is already being extracted.
The variable baseFret contains the fret number that is found on the right of the diagram.
Using the variable fret in the for loops you can get the fret position of the fingers by calculating baseFret+fret. The string can be extracted from the variable string.

Edit: I just remembered, it is not really necessary, because that information can be extracted fromt he HTML code:

<img src="/images/chordshape/Dsharp-Major-Dsharp-11%2C13%2C13%2C12%2C11%2C11-sharps-finger.png" alt="D# chord {11 13 13 12 11 11} chord" class="img-responsive" width="200" height="200">

We can simply extract the {11 13 13 12 11 11} part to get the positions.

So yeah, the main problem right now is that the "2" is being detected as a "1". Once that is solved, I could parse all the data.

Edit2:
Correction: To get the string and fret number inside the for loops you'd have to use the i and j actually. i being the string (0-5) and j being the relative fret number. The actual fret number would be baseFret+j then.

[T-vK]

Okay, I think I have everything I need now. I managed to get it to recognize the numbers correctly now.
I've pretty much already written the whole code that is necessary to extract all the data. I just need to add error handling because with 10000 http request, some are bound to fail and I'd rather not start from scratch every time that happens.

My code so far:

#!/usr/bin/env node
const { createWorker, SetVariable } = require('tesseract.js') // npm i tesseract.js@next
const sharp = require('sharp')
const rp = require('request-promise-native')
const cheerio = require('cheerio')
const fs = require('fs-extra')
const download = require('download')

const digitWorker = createWorker({
  //logger: m => console.log(m)
})
const numberWorker = createWorker({
  //logger: m => console.log(m)
})


async function imageToNumber(image) {
    /*const result = await Tesseract.recognize(image, {
        lang: 'eng',
        //tessedit_pageseg_mode: '10', // singe char mode
        tessedit_char_whitelist: '0123456789T' // allow letter T and numbers
    })
    Tesseract.terminate()
    return parseInt(result.text)*/
    const { data: { text } } = await numberWorker.recognize(image)
    return parseInt(text)
}
async function imageToDigit(image) {
    const { data: { text } } = await digitWorker.recognize(image)
    return parseInt(text)
}

const fingerWidth = 15
const fingerHeight = 15
const stringPositions = [18,43,68,93,118,143]
const fretPositions = [55,80,105,130,155]

const baseFretPositionX = 162
const baseFretPositionY = 51
const baseFretPositionWidth = 24
const baseFretPositionHeight = 24


const stringNames = 'EADGBe'

const jGuitarBaseUrl = 'https://jguitar.com'

async function getChordDiagramBaseFret(inputImg) {
    const tmpOutput = await inputImg.clone().extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toBuffer()
    const baseFret = await imageToNumber(tmpOutput)
    //await inputImg.clone().extract({ left: baseFretPositionX, top: baseFretPositionY, width: baseFretPositionWidth, height: baseFretPositionHeight }).toFile(`extracted - Base Fret: ${baseFret}.png`)
    return baseFret
}

async function getChordDiagramFingering(inputImg) {
    let tmpOutput
    const fingering = []

    for ([string,stringPos] of Object.entries(stringPositions)) {
        let fingerFound
        let fingerFoundFret
        for ([relFret,fretPos] of Object.entries(fretPositions)) {
            tmpOutput = await inputImg.clone().extract({ left: stringPos, top: fretPos, width: fingerWidth, height: fingerHeight }).toBuffer()
            const finger = await imageToDigit(tmpOutput)

            //await inputImg.clone().extract({ left: stringPos, top: fretPos, width: fingerWidth, height: fingerHeight }).toFile(`extracted - String: ${string} (${stringNames[string]}) - Relative Fret: ${relFret} - Finger: ${finger}.png`)
            if (!isNaN(finger)) {
                fingerFound = true
                fingering.push(finger)
                break
            }
        }
        if (!fingerFound) {
            fingering.push(0)
        }
    }
    return fingering
}


const rootNotes = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
const bassNotes = rootNotes
const chordSuffixes = [
  "Major",
  "Minor",
  "Diminished",
  "Augmented",
  "Suspended 2nd",
  "Suspended 4th",
  "Major Flat 5th",
  "Minor Sharp 5th",
  "Minor Double Flat 5th",
  "Suspended 4th Sharp 5th",
  "Suspended 2nd Flat 5th",
  "Suspended 2nd Sharp 5th",
  "7th",
  "Minor 7th",
  "Major 7th",
  "Minor Major 7th",
  "Diminished 7th",
  "Augmented 7th",
  "Augmented Major 7th",
  "7th Flat 5th",
  "Major 7th Flat 5th",
  "Minor 7th Flat 5th",
  "Minor Major 7th Flat 5th",
  "Minor Major 7th Double Flat 5th",
  "Minor 7th Sharp 5th",
  "Minor Major 7th Sharp 5th",
  "7th Flat 9th",
  "6th",
  "Minor 6th",
  "6th Flat 5th",
  "6th Add 9th",
  "Minor 6th Add 9th",
  "9th",
  "Minor 9th",
  "Major 9th",
  "Minor Major 9th",
  "9th Flat 5th",
  "Augmented 9th",
  "9th Suspended 4th",
  "7th Sharp 9th",
  "7th Sharp 9th Flat 5th",
  "Augmented Major 9th",
  "11th",
  "Minor 11th",
  "Major 11th",
  "Minor Major 11th",
  "Major Sharp 11th",
  "13th",
  "Minor 13th",
  "Major 13th",
  "Minor Major 13th",
  "7th Suspended 2nd",
  "Major 7th Suspended 2nd",
  "7th Suspended 4th",
  "Major 7th Suspended 4th",
  "7th Suspended 2nd Sharp 5th",
  "7th Suspended 4th Sharp 5th",
  "Major 7th Suspended 4th Sharp 5th",
  "Suspended 2nd Suspended 4th",
  "7th Suspended 2nd Suspended 4th",
  "Major 7th Suspended 2nd Suspended 4th",
  "5th",
  "Major Add 9th"
]

function getChordImgsFromHtml(html) {
    const $ = cheerio.load(html)

    const chordShapeImgs = []

    $('img.img-responsive').each(function() {
        chordShapeImgs.push(this)
    })

    return chordShapeImgs
}

function getPageInfoFromHtml(html) {
    const $ = cheerio.load(html)
    let [fullMatch, firstChordIndex, lastChordIndex, totalChordCount] = $('body').html().match(/Showing results (\d+) to (\d+) of (\d+) chord/)
    firstChordIndex = parseInt(firstChordIndex)
    lastChordIndex = parseInt(lastChordIndex)
    totalChordCount = parseInt(totalChordCount)
    const totalPageCount = Math.ceil(totalChordCount/lastChordIndex)
    return {
        firstChordIndex,
        lastChordIndex,
        totalChordCount,
        totalPageCount
    }
}

async function main() {
    await digitWorker.load()
    await digitWorker.loadLanguage('eng')
    await digitWorker.initialize('eng', 2)
    await digitWorker.setParameters({
        'tessedit_char_whitelist': '0123456789'
    })
    await numberWorker.load()
    await numberWorker.loadLanguage('eng')
    await numberWorker.initialize('eng', 0)
    await numberWorker.setParameters({
        'tessedit_char_whitelist': '0123456789'
    })

    /*
    const inputImg = await sharp('test.png')

    const fingering = await getChordDiagramFingering(inputImg)
    const baseFret = await getChordDiagramBaseFret(inputImg)
    console.log('Base Fret:', baseFret)
    console.log('Fingering detected:', fingering)
    process.exit()
    */

    for (const rootNote of rootNotes) {
        for (const chordSuffix of chordSuffixes) {
            for (const bassNote of bassNotes) {
                const encRootNote = encodeURIComponent(rootNote)
                const encChordSuffix = encodeURIComponent(chordSuffix)
                const encBassNote = encodeURIComponent(bassNote)
                const url = `${jGuitarBaseUrl}/chord?root=${encRootNote}&chord=${encChordSuffix}&bass=${encBassNote}&labels=finger&gaps=0&fingers=4&notes=sharps`

                const html = await rp({ uri: url })
                let chordShapeImgs = getChordImgsFromHtml(html)

                const { firstChordIndex, lastChordIndex, totalChordCount, totalPageCount } = getPageInfoFromHtml(html)
                for (let i=2; i<=totalPageCount; i++) {
                    let nextPageUrl = `${url}&page=${i}`
                    const html = await rp({ uri: nextPageUrl })
                    const newChordShapeImgs = getChordImgsFromHtml(html)
                    chordShapeImgs = chordShapeImgs.concat(newChordShapeImgs)
                }

                for (let [i,img] of Object.entries(chordShapeImgs)) {
                    i = parseInt(i)
                    let [fullMatch, fingerPositions] = img.attribs.alt.match(/\{([0-9x]+\s[0-9x]+\s[0-9x]+\s[0-9x]+\s[0-9x]+\s[0-9x]+)\}/)
                    fingerPositions = fingerPositions.split(' ')
                    const imgUrl = `${jGuitarBaseUrl}${img.attribs.src.replace(/\/\//g,'/')}`

                    let fullChordName
                    if (rootNote === bassNote)
                        fullChordName = `${encRootNote}${encChordSuffix}`
                    else
                        fullChordName = `${encRootNote}${encChordSuffix}${encodeURIComponent('/')}${encBassNote}`

                    const filePath = `./download/diagrams/${fullChordName}.png`
                    await download(imgUrl, `./download/diagrams/`, {filename: `${fullChordName}.png`})

                    const inputImg = await sharp(`./download/diagrams/${fullChordName}.png`)
                    const fingering = await getChordDiagramFingering(inputImg)

                    console.log('fullChordName', fullChordName)
                    console.log('fingerPositions', fingerPositions)
                    console.log('fingering', fingering)
                    process.exit() // cancel early because we only want to parse one chord diagram for now

                }

                process.exit()
            }
        }
    }
    process.exit()
}

main().catch(console.error)

I also need to add a bit more code to extract the "subsets" for each chord (which unfortunately don't have fingerings) and convert the chord names a bit (e.g. instead of CMajor and CMinor, it should just use C and Cm).

I let the script run and extract the first chord from jguitar completely by itself and it came up with this data:

fullChordName CMajor
fingerPositions [ 'x', '3', '2', '0', '1', '0' ]
fingering [ 0, 3, 2, 0, 1, 0 ]

From
<img src="/images/chordshape/C-Major-C-x%2C3%2C2%2C0%2C1%2C0.png" alt="C chord {x 3 2 0 1 0} chord" class="img-responsive" width="200" height="200"><div class="row">

[szaza]

Wow, cool! Nice and interesting work!
Just an idea, but maybe we could use the chords-db as validation set. There wouldn't be a perfect match because chords-db doesn't contain all the chords available on the jGuitar, however we could check manually the differences. Maybe, we could check only the most common occurrence of the chord.

Probably, for simplified suffixes we could use what we have in the chords-db here: https://github.com/tombatossals/chords-db/blob/master/src/db/guitar/suffixes.js

For generating slash chords we could reuse the array of rootNotes ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'] as base notes, so we would have a three level deep for loop, e.g.: rootNote + suffix + base note => C + m + '/' + A => Cm/A.

Can we parse also barres chords?
Could you create a new repository with your code and add me as contributor?

[T-vK]

Look closely, I'm already doing the three level deep loop to really get all chords ;)

    for (const rootNote of rootNotes) {
        for (const chordSuffix of chordSuffixes) {
            for (const bassNote of bassNotes) {

The script can indeed also parse barre chords.

Sure, I'll create the repo later today.

[szaza]

All right, many thanks!

[T-vK]

@szaza https://github.com/T-vK/jguitar-parser

There are a few TODOs in the file:
https://github.com/T-vK/jguitar-parser/blob/master/index.js#L260
https://github.com/T-vK/jguitar-parser/blob/master/index.js#L269

and in addition to that I need to figure out what order the shapes of each chord should be sorted. I guess the subsets should probably be added to the very end as there is no fingering information for those.
I also wonder if it would make sense to merge the jguitar chords with the data I extracted from chordsdatabase.com and the data from this repository. Because jguitar may have all the chord shapes, but only has one fingering for each chord and for the subsets it doesn't have any fingerings at all.

The way I'm currently parsing the chord diagrams is not very efficient. All in all it takes about 1 second for each chord diagram. So extracting and parsing all the chords would take about 30 hours. But, I think I'm not gonna bother with making it more efficient.

[szaza]

Hi @T-vK, many thanks for your effort, great work!
I think the 1 second runtime/chord is not a problem, because in best case we have to run it only once for the full dataset. Also we can run in parts for smaller datasets to make validations.

[tombatossals]

That's fantastic job @T-vK. We could parse the chords and make some validations using chords-db and manually as @szaza says.

I'm working too on the gh-pages of react-chords, the idea is accomplish a basic raw visualizator of all the chords that can help us to detect problems with the news imported chords.

Just tell me if you need help with the parsing

[T-vK]

It seems like some chords are too crazy to be played on the guitar.
C/F# Minor Major 9th for instance doesn't have a result on jguitar: https://jguitar.com/chord?root=C&chord=Minor+Major+9th&bass=F%23&labels=finger&gaps=2&fingers=4&notes=sharps

Edit:
But I also have good news. I found a way to get the fingerings for the subsets. :)
If you remove the -true from the URL, you get the fingerings:
So instead of:
https://jguitar.com/images/chordshape/C-Major-C-x%2C3%2C2%2C0%2Cx%2C0-sharps-finger-true.png
We need:
https://jguitar.com/images/chordshape/C-Major-C-x%2C3%2C2%2C0%2Cx%2C0-sharps-finger.png

Edit2:
I'm not so sure anymore if jguitars data is good enough. This one for instance: https://jguitar.com/images/chordshape/C-Major-C-8%2Cx%2C10%2C9%2C8%2C8-sharps-finger.png

The fingering seems very unrealistic. At least I don't think that I could play it this way. I would probably use my thumb on the low E and A instead of my index finger. What do you think?

Edit3:
I also found this interesting script: https://jguitar.com/js/chordcalculator.cf36bb7e77075a1e6a4c33d9fee9a608.min.js

It might be worth it to reverse engineer. It seems to be responsible for calculating the subsets and might be able to do even more.

[szaza]

"The fingering seems very unrealistic. At least I don't think that I could play it this way. I would probably use my thumb on the low E and A instead of my index finger. What do you think?" - I also observed that sometimes fingering seems unrealistic.

@T-vK I would like to convert the output of your application according to chords-db format in order to be able to compare with the validation set and later to extend chords-db. Are you still working on the jguitar-parser? If it is ok with you then I'm going to create a new development branch and implement the needed changes there. Please let me know if you are already working on similar feature.

[T-vK]

@szaza Feel free to go for it. I just pushed what I got so far. I'm currently running the parser to download all the raw data from jguitar into the cache. I think I'm not gonna add the subsets to the output for now because too many of them seem to be useless.

I already have 40% of the raw jguitar data downloaded and I expect the final output json to be ~10MB in size. But that's with the long chord names. One we have a function to generate the shorter more common chord names it will probably be a little bit smaller.

[szaza]

Ok, then I'm going to implement a converter that converts the output of your application to chords-db format and probably I can solve the chord names problem too. If you think I can parse a part of the jguitar chords with your tool and we could merge the output at the end, however I could do it only tomorrow afternoon. Please let me know if I can help anything else.

[T-vK]

I'm going to implement a converter that converts the output of your application to chords-db format and probably I can solve the chord names problem too

That would be great!

If you think I can parse a part of the jguitar chords with your tool

Thanks, but it'll be easier for me when I have all the raw html and png data on one computer. I'll probably have to parse all that data again anyway at some point.

[szaza]

Hi,
The implementation of the converter is mainly done. I implemented it as a subproject: https://github.com/T-vK/jguitar-parser/tree/chords-db-converter; Please review it when you have some free time. At the moment it is not able to generate information about barres and capo.
In the chords-db there are chords where barres and capo attributes appear, for example:

    {
      frets: '575688',
      fingers: '131244',
      barres: [5, 8],
      capo: true
    },

I cannot find any information regarding to the barres and capo in the output of the jguitar-parser tool.
Is that information missing from the generated chords.json?

[T-vK]

Please review it when you have some free time.

Awesome, I will!

Is that information missing from the generated chords.json?

I don't really see the value of the barre/capo information. But I would say if the same finger has to be positioned on multiple strings on the same fret, then you have a barre. And if the index finger makes a barre, then you could alternatively use a capo instead of the index finger and play it as an open chord. In that case, however, the fingering would most likely be very different. But I guess that's when you would want to set the capo property to true.

I'm almost done scraping the data btw. It crashed a couple of times, but I'm at 95% now.

[tombatossals]

Hi, I have finished a basic chord visualization tool that can help us to review the new chords imported from a parser: https://tombatossals.github.io/react-chords

Maybe some capo/barre info can be added to the imported set manually. The test could detect basic typos, and we can review the import and improve the queality of the chord in the JSON with more properties.

Great job guys!

[T-vK]

Okay, it's done. The final json file ist about 13.5MB in size. I also created a smaller version that only contains one shape per chord which is about 1.2MB in size. I've uploaded the files into this repo: https://github.com/T-vK/chord-collection

Now we need to verify that the data really is good enough.

[tombatossals]

That's great @T-vK. I'm going to work in an import tool based on your 13.5MB JSON which will merge the new chords with the actual chords-db structure. My idea is to mantain the already existing chords in the DB, and add the new chords that does not exists.

After that, some basic testing and some visually review of them and the job will be done.

What do you think guys?

[szaza]

@T-vK "I would say if the same finger has to be positioned on multiple strings on the same fret, then you have a barre. And if the index finger makes a barre, then you could alternatively use a capo" - yap, I was thinking the same, I just wanted to make sure that there is no barres and capo info exported. I though before that maybe I forget to set properly some settings in your tool. It should be a peace of cake to make this modification in the converter tool, however today and tomorrow I won't be in front of my laptop. If you have some free time sooner, you can implement the required changes, otherwise I'll implement it when I'm back.

@tombatossals good idea. I also would extend the current chords-db format with the possibility to add more variations for fingerings, cause @T-vK tools supports multiple fingerings. After the new structure is done, we could modify the converter accordingly.
I was thinking something like:

 {
    frets: '575688',
    fingers: [ '131244', '123456', '789abc'],
    barres: [5, 8],
    capo: true
}

So, the fingers attribute would be an array. Maybe, it would be also good to create a REST based webservice that provides chord variations in JSON format. It would work similarly like the jGuitar, however it would return chords in JSON format.

What do you think about it?

[tombatossals]

Sure! We could extend the structure so more information make chord rendering richer to see. The change you propose in the fingers property could be easily ported to the actual chord database.

About the REST service i think it's more related to the custom implementation of the visualization tool. It could be useful for a light web application, but I suppose a developer would prefer to manage their own chord query service (REST, GraphQL) that depend on a 3rd party service. The greatness of this DB is that is open so anyone can use their own solution, because of the easy of parsing of the chords.

I have been browsing JGuitar and the database size is huge (in a lot of tunings). But I think they are using a computer-based calculation of the chord positions. Our advantadge to manage manually the database is that we can add fingering, a think not possible to calculate with a computer (I think so).

[T-vK]

I think an (open source) REST service would be a nice addition. Because forcing a user to download 13.5MB is bound to end in a bad user experience. Even the smaller 1.2MB file is way too big considering that in many areas you only get a 2G mobile connection. Not to mention that many people around the world have very small data caps like 100MB/month.
I don't think there is a free way to run a server like this though, so everyone who needs it would have to run their own.

But maybe there is a way to work around that by splitting the json data into many small files. Maybe one file per chord. Then we could host all these files on Github and write a JavaScript library that could automatically get the chord files that a website wants to load.

[szaza]

I think it is possible to host it for free. Currently, I host a web-page on Google Cloud services with low resources and in my case I only have to pay for the Database Management System. In our case we wouldn't need any RDBMS and we could serve chords directly from the JSON files. I could create a free REST based web-service.

[T-vK]

What if a big website with millions of users per day integrates the REST service, causing massive network traffic? Would that still be considered "low resources"?
I also wonder about the domain name, is Google offering that for free as well?

[szaza]

"What if a big website with millions of users per day integrates the REST service, causing massive network traffic? Would that still be considered "low resources"?" - no, obviously that wouldn't be feasible for free, however Google Cloud services can be scaled easily. If a big website with millions of users per day would integrate our REST API than most probably it is valuable enough for them to support us in order to scale up our service. Obviously without support we cannot create a robust webservice, but at least we could start it with low resources and make it accessible.

"I also wonder about the domain name, is Google offering that for free as well?" - no, that also wouldn't be available for free, however I have a valid domain the guitar-chords.org and this service could run from a subdomain like api.guitar-chords.org.

[szaza]

I've just added the barres support for the converter, however I found some particular cases e.g.: Dm/G#

{
  "positions":["4","5","x","7","6","5"],
  "fingerings":[["1","2","4","4","1","2"]]
}

If I understand well it means that index finger is used both for the 4th and 6th frets.
There are several more similar cases. Could you check them please?
I also couldn't find this chord with this fingerings in the JGuitar website.

[T-vK]

I mean.. it wouldn't hurt to have a REST service, but I think it would be a little naiive to assume that a big site using the service would support it financially (and that for every month). I think I'm more leaning torwards the pure GitHub solution with one json file per chord and potentially a JS library to make the access as easy as possible.

There is most likely a bug in the image recognition. I'll try to find some time tomorrow to check.

I assume the problem is the barre bow that is inside the diagram rather than on top of it:
https://jguitar.com/images/chordshape/D-Minor-Gsharp-4%2C5%2Cx%2C7%2C6%2C5-sharps-finger.png

Maybe I should move away from the ocr approach and just straight up check if the image contains specific pixels at specific positions instead.

[szaza]

"it would be a little naiive to assume that a big site using the service would support it financially" - :-)) believe me if it is important enough for them, they would support our service.

"I think I'm more leaning torwards the pure GitHub solution with one json file per chord and potentially a JS library to make the access as easy as possible." - that is also a good solution. Just check my converter, it splits up the generated json file into smaller parts by base notes and suffixes as it is in the chords-db structure. You can modify it very easily to split the generated json file and keep your format. So, creating JS library on Github is totally fine for me, actually by extending chords-db we do the same.

"Maybe I should move away from the our approach and just straight up check if the image contains specific pixels at specific positions instead." - I agree, comparing detected image parts with raw images is a good idea.

[tombatossals]

Hi guys, I think that a website where people could review and add new chords, and maybe compose them into songs could be of interest of the community, but only a service to obtain the chord (when the full database is freely accesible) I think will get no intereset.

BUT, providing the JSON file of the chord maybe needs no REST service. Take a look at this page, it's generating the SVG version of the chord when you click in a chord:

https://tombatossals.github.io/react-chords/guitar/C/alt

Generating the JSON file, a PNG file, etc. could be as easy as that. It could be hosted in Github (free and high availability) and anyone could get the version of the chord by simple traversing an URL (the URL is statically composed based in the key and suffix).

Do you think it's useful for your idea? Instead of querying a REST service, it only needs to access a URL path and you get the chord in SVG, PNG, JSON format.

In any case, I'm importing the chords into our dataset, I have created a "newdb" branch and good news, it's almost working. I have passed some tests into it and some chords fail to pass because:

"The 1 position frets array should have at most 4 fingers of distance"

I considered when was coding the tests that no chord should have 4 fingers of distance from one finger to another. For example the "Cmmaj9/G#" has this problem. The JGuitar.com site says that this chord is impossible. Maybe it has been bad imported?

[T-vK]

Yes, that's what I meant and having images would of course be a nice addition.

If you adjust the settings on jguitar it will give you quite a few possibilities: https://jguitar.com/chord?root=C&chord=Minor+Major+9th&bass=G%23&labels=finger&gaps=2&fingers=4&notes=sharps
I'm not sure what you mean by "4 fingers of distance", but I would agree that these shapes may not be possible to play. It would be hard to accurately determine what is possible and what not though. (Some fingers can be stretched apart much further than others and every fret has a different width...)

If you google for guitar chord fingering algorithm you find quite a few interesting papers and discussions though. Maybe there is a complete algorithm somewhere out there that we could use to calculate the fingerings.

[szaza]

Hi guys,
Today, I created a split version of @T-vKs chord collection, however it keeps the format of David Ruberts (@tombatossals) chords-db. I also created a script that excluded 19305 chords that most probably impossible to play. I also compared the common parts of chords-db and the newly generated collection and there were few differences. I was curious whether chords-db is a valid subset of @T-vKs collection and it seems that it is. Almost every positions, which are present in chords-db takes part correctly of @T-vK s collection. Of course it doesn't prove that every positions in @T-vKs collection is correct, however it shows that the parser script worked well.
The collection is accessible here: https://github.com/szaza/guitar-chords-db-json;

[tombatossals]

Hi @szaza good work, I have been working too on that but with the smaller version in the "newdb" branch of this repository. Finally I was able to join the visualizator to this branch here:

https://tombatossals.github.io/chords-db/

It misses the "barres" and "capo" information.

[tombatossals]

If you google for guitar chord fingering algorithm you find quite a few interesting papers and discussions though. Maybe there is a complete algorithm somewhere out there that we could use to calculate the fingerings.

Really impressive the chord fingering algorithm. Maybe an algorithm can be written that could create any chord without any previous declaration like we're doing, including the fingering. That would be really efficient.

[T-vK]

@tombatossals I'm sure this is possible. Generating the chords itself would probably be relatively simple. But filtering out the impossible ones is very difficult. Although I think using some machine learning it could actually be pretty easy. I have no experience with that though...

Edit:
Maybe tonaljs could be used to find the notes that each chord consists of. Then it would just be a matter of mapping out where these notes can be found on the fret board to find every possible (and impossible) fingering.

Edit2:
The chordShapes function of this looks promising: https://www.npmjs.com/package/tonal-fretboard