Skip to content

Preprocessing errors

Jump to bottom
Kalinda Pride edited this page Jul 27, 2023 · 29 revisions

This page is a guide to some of the errors you might see in the terminal while using LingView, and what you can do about them. If you got an error, try searching for it on this page! If you're unable to resolve the error after following these steps, contact [email protected] for help, and include the error message in your email.

Terminological note: Nowadays, when you want to rebuild LingView after changing any ELAN/FLEx/media/jsx files, it's recommended to run npm run quick-build. That command is a shorthand for node preprocessing/rebuild.js followed by npm run webpack. If you got an error from quick-build, it will be described in this document as coming from either rebuild.js or webpack.

ERROR: no linked audio or video

When running preprocessing/rebuild.js, you might see output like the following:

🚨  WARN: kuke_chiste.eaf is missing correctly linked audio. Attemping to find link...
❌  ERROR: Cannot find matching audio for kuke_chiste.eaf.
❌  ERROR: kuke_chiste.eaf has no linked audio or video in the media_files directory. It will be processed as an untimed file and no audio, video, or time alignment will be displayed on the site.

This means that LingView thinks there ought to be an audio or video file for the kuke_chiste.eaf file, but it wasn't able to find one.

To fix this, put the .wav, .mp3, or .mp4 file in the media_files directory and rename it to match the ELAN file's (or FLEx file's) name, for example, kuke_chiste.mp3. Then run the preprocessing/rebuild.js script again.

If you'd rather not use a matching file name, there are two other ways to fix the error. In either case, start by putting your audio or video file in the media_files directory. Then option 1 is to open the ELAN file, choose Edit and then Linked Files from the menu, and locate the file you just placed in the media_files directory. Option 2 is to use the edit.js script, which will prompt you for the name of your audio or video file. Finally, after either option, run the preprocessing/rebuild.js script again.

If you don't want to give LingView an audio or video file right now, you can ignore the error and everything else will still work.

data/index.json doesn't exist

You might see this error the first time you run webpack. To fix it, run node preprocessing/rebuild.js, which creates the index.json file.

Errors with third-party software ("dependencies")

LingView depends on several publicly available software packages (called "modules" or "dependencies") to complete certain tasks. When running a LingView script for the first time, you might see an error message that one of these dependencies is "undefined" or that says "cannot find module". This indicates that the installation of LingView on your computer was incomplete or incorrect.

LingView's dependencies include fs, util, xml2js, prompt, inquirer, tabletojson, and sync-rpc.

To fix this, download Node and NPM (if you haven't already), then run npm install, following the setup instructions.

npm ELIFECYCLE

When running npm run webpack, you might see the following error message:

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] webpack: `webpack-cli --config webpack.config.js`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] webpack script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

It may be accompanied by other error messages like:

ERROR in ./jsx/App/Stories/Story/Display/Sentence.jsx
Module not found: Error: Can't resolve 'html-es6cape'

This indicates that the installation of LingView on your computer was incomplete or incorrect.

To fix this, download Node and NPM (if you haven't already), then run npm install, following the setup instructions.

npm cb.apply is not a function

If you install npm while another copy is already installed, npm might break. If so, whenever you try to run a command that starts with npm, you'll get the error message npm ERR! cb.apply is not a function, and npm won't actually run your commands. (If you're curious, see the Stack Overflow Question and/or the GitHub issue.)

To fix it, if you're on Windows:

  1. Go to C:\Users(your username)\AppData\Roaming.
  2. Delete the npm folder and, if there is one, the npm cache folder.
  3. Run npm cache clean --force.

If you're on Linux, just run npm cache clean, or if that doesn't work, npm cache clean --force.

WARNING in size limit

When running npm run webpack, you might see the following warnings:

WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
...

WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
...

You can ignore these warnings. LingView is pretty big, especially when used with large audio or video files, but we haven't noticed bad impacts on web performance.

SyntaxError: Unexpected number or token in JSON

When running the preprocessing/rebuild.js script, you might see output like the following:

Building database...
undefined:87
}rt_time_ms": 0,
 ^
SyntaxError: Unexpected token r in JSON at position 1772
    at Object.parse (native)
    at Object.build (C:\Users\Kalinda\Desktop\GitHub\LingView\preprocessing\build_database.js:12:29)
    at whenDone (C:\Users\Kalinda\Desktop\GitHub\LingView\preprocessing\rebuild.js:20:8)
    ...

This error is rare and happens more or less at random. The current theory is that it happens when LingView tries to access the same file from two different places at once, resulting in a garbled JSON file.

To fix it, simply run the preprocessing/rebuild.js script again.

TypeError: Cannot read property of undefined

While running any of the preprocessing scripts, you might see output like the following:

events.js:160
      throw er; // Unhandled 'error' event
      ^
TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
    at getParagraphSentences (C:\Users\Kalinda\Desktop\LingView\preprocessing\flex_utils.js:21:31)
    at Object.getDocumentFirstWord (C:\Users\Kalinda\Desktop\LingView\preprocessing\flex_utils.js:5:25)
    at preprocessText (C:\Users\Kalinda\Desktop\LingView\preprocessing\preprocess_flex.js:303:50)
    ...

or more generally, any error message that says TypeError and undefined, and which doesn't mention any of LingView's dependencies (see previous section).

This error indicates an unsupported data format. For example, the only supported formats for FLEx files are Verifiable Generic XML and Flextext. If you export your FLEx file using another format, such as Microsoft Word XML, you will encounter this error when you run the preprocessing/rebuild.js script.

This error might also be caused by a bug in LingView. We've tested LingView on a variety of FLEx and ELAN files, but it's always possible that we missed something. For example, we didn't do extensive testing for part-of-speech tags or other closed-class annotations, so LingView might error when processing some FLEx or ELAN files that include them.

A similar-looking error is caused when your Node.js version is out-of-date. This error is actually related to LingView's dependencies, not to an unsupported FLEx or ELAN format:

TypeError: Cannot read property ‘readdir’ of undefined
    at Promise.all.then (C:\Users\Kalinda\Desktop\GitHub\LingView\preprocessing\rebuild.js:64:23)
    at <anonymous>
    ...

First steps:

  • Check your Node.js version by running node --version in your terminal. If it's less than 10.0.0, download the Node installer and run it. If you're using nvm, you also need to tell nvm to use the newly-installed version, for example, nvm use 16.1.0. Finally, run npm install.
  • Use the error message to identify which file type is causing the error. In this example, the first file is preprocessing\flex_utils.js, indicating a problem with FLEx rather than ELAN. The function names at the beginning of each line (getParagraphSentences, Object.getDocumentFirstWord, etc.) might also mention either FLEx or ELAN.
  • If the error occurred while running the preprocessing/rebuild.js script, note that the problem might be with any of the files being processed, not necessarily the first or last filename that was printed out.
  • Check for incorrect file extensions. The data/elan_files directory should contain only .eaf and .pfsx files, and the data/flex_files directory should contain only .xml and .flextext files. Remove any files with incorrect file extensions, or replace them with the correct data type by re-exporting from FLEx. Then run the script again to see if that fixed the problem.
  • You may need to remove or replace files that have the correct file extension, but which use the wrong format internally, such as FLEx files that were exported using the Microsoft Word XML format.

If all else fails ("deep clean" method):

  1. Copy your data directory elsewhere, to avoid losing anything.
  2. Empty out your data/flex_files and data/elan_files, and replace your data/index.json and data/database.json with the ones from the BrownCLPS/LingView repository on GitHub.
  3. Run preprocessing/rebuild.js so that LingView recognizes the added files, then run the error-causing script (if different from rebuild.js). If you get an error this time, then the problem is something else, not one of your FLEx or ELAN files. Please email [email protected] for help, and include the error message in your email.
  4. Add back your FLEx and ELAN files one at a time, re-running preprocessing/rebuild.js and then the error-causing script after each one. When you add an ELAN file, add its .pfsx file at the same time, if there is one.
  5. When the error comes back, stop and examine the file you just added. If it's an ELAN file: Open it in ELAN. If you can successfully open, view, and edit it, then your file is fine and this is a bug in LingView; otherwise, your file got corrupted somehow and you will need to re-create or fix it. If it's a FLEx file: re-export it from FLEx, making sure to choose the "Verifiable Generic XML" format or the "FLEXTEXT" format. If this doesn't resolve the error, your file is fine and this is a bug in LingView.
  6. If you uncovered a LingView bug in the previous step, please email [email protected] to report it. Include the error message and, if possible, the file that caused the error.
  7. After resolving the error, you'll need to delete the sample files 103.xml, 20170804_kuke_chiste_FACQ.eaf, 20170804_kuke_chiste_FACQ.mp3, and 20170804_kuke_chiste_FACQ.mp3, then run preprocessing/rebuild.js again. You'll also need to re-add any metadata (date, genre, etc.) that you previously added through LingView.

No error, but files are missing

There is no error message for this. If this problem affects you, you'll notice some the FLEx or ELAN files from your data directory are omitted from the Index of Texts page of your site, even after npm run quick-build.

LingView relies on the unique 36-character ID that ELAN or FLEx assigns to each file, such as 97b8ab3b-d2a5-428a-aa68-0aa304ba1c44, and it assumes the ID should be different for each file. When two ELAN or FLEx files have the same ID, only one of them can be included in the site.

View the ID by opening the .eaf file directly on GitHub and checking the "URN" property near the beginning. Then, edit the URN value by replacing some of the 0-9 and a-f characters with other ones, and npm run quick-build again. The edited file will now be included in the site.

Clone this wiki locally