override.js file no longer overriding user.js file in FF58+

[overdodactyl]

edit: 1413413 is the cause - Thorin

edit II: I submitted a bug report report (1416279) to hopefully address this - overdodactyl

edit III: bug was labeled WONTFIX

The code that implements this functionality has been removed. After sending an intent to remove email to dev.platform and discussing, we decided that it wasn't worth the effort of continuing to support as we re-implement the preference service.

edit IV: wiki link

---

Greetings,

In the past, I've been using an override.js file to override some preferences from user.js as suggested in Section 1.5 of the wiki.

Sometime within the past week (maybe two?), the settings in my override.js file are no longer being used to override the user.js file in Firefox Nightly.

I can reproduce this issue in a new Nightly profile, however, if I copy my user.js file and my preferences folder containing override.js to a new profile and use the stable version of Firefox, I do not experience this issue.

Has something changed? Can anyone else confirm they are experiencing this in Nightly (or, alternatively, that they aren't having the problem)?

I apologize if this has already been addressed...I did a quick search here and on bugzilla and couldn't find anything relevant .

[overdodactyl]

Thanks for the reply @Thorin-Oakenpants, I'll take a closer look at bugzilla and report back with anything I find!

Hopefully it wasn't removed on purpose...I've been relying on that :)

[overdodactyl]

Thank you @Thorin-Oakenpants!! I appreciate the advice and you looking into that...I'm not super familiar with bugzilla, so any new tips like that are really helpful :) I'm glad a tickets already been submitted (one that I obviously missed).

I don' have too many overrides either, so throwing them at the end of user.js is a good solution in the mean time.

Thanks again!

[overdodactyl]

Thanks for updating the wiki page!

I noticed that was for Thunderbird...I guess I just naively thought the ticket might include Firefox haha. You're right though, that doesn't look super promising.

I can go ahead and submit a new ticket if you think it would be helpful.

[overdodactyl]

I went ahead and submitted a bug report this morning 1416279 for anyone that wants to follow it.

Edit: Was labeled as WONTFIX...I guess there's no hope after all :(

The code that implements this functionality has been removed. After sending an intent to remove email to dev.platform and discussing, we decided that it wasn't worth the effort of continuing to support as we re-implement the preference service.

[overdodactyl]

No problem @Thorin-Oakenpants, sorry I couldn't do more! If it helps anyone else out there, a viable alternative (at least for me) is to use something like the following script:

#!/bin/bash
curl -o .../profile/user.js https://raw.githubusercontent.com/ghacksuserjs/ghacks-user.js/master/user.js
cat .../profile/preferences/override.js >> .../profile/user.js 

it at least quickens the process of keeping up-to-date and could be easily modified to use various *.js files.

[earthlng]

Hi @overdodactyl

sorry I couldn't do more!

you found a problem, informed us about it, reported it to mozilla and provided a script that can be used as a workaround - what more could we ask for! Thanks for taking the time to do all that, much appreciated! ❤️❤️❤️

[claustromaniac]

I made myself this batch script (link removed) for Windows. I leave it here in case anyone wants it.

It backs up user.js and then it downloads the user.js file from this repository and appends user-overrides.js to it.

It goes in the profile folder alongside user.js and user-overrides.js.

Thank you for this great resource, by the way. I've been using user.js files for a long time but this one is by far the most comprehensive I ever had.

Also thank you @overdodactyl for the heads up.

[claustromaniac]

@Thorin-Oakenpants I improved it a bit with your ideas, and I changed a few things.

If user.js.bak already existed it just deleted it and created a new backup, but I changed that. It now creates a secondary backup, and when the update finishes it checks whether user.js and the new backup are the same or not and, if they are, it keeps the old (different) backup. Otherwise it keeps the new backup (and discards the older one).

Other significant change is that now it doesn't rely on the existence of user-overrides.js to update user.js.

I guess there's still room for improvements, so let me know what you think.

Edit: Last-minute minor fix 😓

[overdodactyl]

Hi @earthlng

Thanks for taking the time to do all that, much appreciated!

It's the least I could do given how much I've benefited from this project :)

@Thorin-Oakenpants

I decided to try and make a version of @claustromaniac's script usable for fellow Mac users out there . I tried to incorporate all your suggestions from his first version, but I made this before I saw the most recent one. Feel free to use it, make any suggestions/change it however you like or completely ignore it all together. I'm pretty new to writing scripts, so this was a fun use of my time either way :)

#!/bin/bash

cd "`dirname $0`"

echo
echo "This script should be run from your Firefox profile directory."
echo

if [ -e user.js ]; then	
	echo "Your current user.js file for this profile will be backed up and the latest ghacks version from github will take its place."
	echo
	echo "If currently using the ghacks user.js, please compare versions:"
	
	online_version="$(curl -s https://raw.githubusercontent.com/ghacksuserjs/ghacks-user.js/master/user.js | sed -n '4p')"
	local_version="$(sed -n '4p' user.js)"
	
	echo "	Available online: ${online_version}"
	echo "	Currently using:  ${local_version}"
	
else	
	echo "A user.js file does not exist in the current directory.  If you continue, the latest ghacks version from github will be downloaded"
fi
echo
echo "If a user-override.js file exists in this directory, it will be appended to user.js."
echo
read -p "Continue Y/N? " -n 1 -r
echo
echo

if [[ $REPLY =~ ^[Yy]$ ]]; then
	if [ -e user.js ]; then
		#backup current user.js
		mv user.js "user.js.backup.$(date +"%Y-%m-%d_%H%M")"
		echo "Your previous user.js file was backed up: user.js.backup.$(date +"%Y-%m-%d")"
	fi
	
	#download latest ghacks user.js
	echo "downloading latest ghacks user.js file"
	curl -O https://raw.githubusercontent.com/ghacksuserjs/ghacks-user.js/master/user.js
	echo "ghacks user.js has been downloaded"
	
	if [ -e user-overrides.js ]; then
		echo "user-overrides.js file found"
		cat user-overrides.js >> user.js
		echo "user-overrides.js has been appended to user.js"
	fi
else
	echo "Process aborted"
fi

[earthlng]

Awesome stuff guys!! Thanks @claustromaniac + @overdodactyl ! ❤️

nits:
In the mac (+linux?) version you're downloading the file twice (not that it matters - just sayin' :) and in certain places it would be better to use && to make sure the previous command succeeded. I would also store the backup filename in a variable so that you can show the actual filename:

#backup current user.js
bakfile="user.js.backup.$(date +"%Y-%m-%d_%H%M")"
mv user.js "${bakfile}" && echo "Your previous user.js file was backed up: ${bakfile}"

[earthlng]

I tested the Windows batch script and noticed that I got this 1 weird line () in the concatenated user.js due to the BOM header because my user-overrides.js was stored as UTF-8. IDK if that's a problem when FF will parse the file and/or whether cat on Mac/Linux will also create that line when the 2 files are not in the same encoding. Just something to keep in mind I guess and maybe something that we should add a note about somewhere because some people might use UTF-8 because of foreign language comments in their user-overrides.js. The raw user.js on github is 1252 ANSI latin-1 encoded.
Users either need to use ANSI encoding or UTF-8 without the BOM for their user-overrides.js file.

[earthlng]

https://github.com/ghacksuserjs/ghacks-user.js/blob/master/updater.bat
https://github.com/ghacksuserjs/ghacks-user.js/blob/master/updater.sh

This will help users a lot, thanks guys!!

[overdodactyl]

@earthlng, I'm glad I could contribute a bit (even if it was something small)!

Thanks for cleaning up my code a bit, that was my first script of any real value over a couple lines, so I figured it was far from perfect :)

I noticed in some of my profiles, the script wasn't locating to the correct directory before running.

I changed cd "dirname $0" to cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd and it seemed to work, but I can't completely explain why...

On a totally unrelated note, I noticed there was a comma missing on line 25 after 'media.webspeech.recognition.enable' in the script for bulk resetting preferences. (thought it was worth pointing out but not worthy of it's own issue :) )

edit: my bad, fixed the comma - thorin

[claustromaniac]

@Thorin-Oakenpants I must admit that I also thought about compiling an exe, but that would bring along the hassle of dependencies and such, right? I mean, for a cross-platform approach we would need something like Python. It would be nice if someone could test @overdodactyl's script on Linux, though. Since it's not overly complex it might just work fine on many distros.

@earthlng Good catch. I wouldn't hope for Firefox to be expecting some people using user.js files made out of chunks of text with different encoding, so I guess we're indeed better off just avoiding that 😆 It's good to know that it seems it's just the BOM header that is leaving trash in the resulting file.

[earthlng]

@overdodactyl
fix for the profile detection problem: 500c129
It's based on info I found online but I don't have a Mac and can't test it. Let me know if it doesn't work.

[claustromaniac]

So... I made some further changes and additions to the batch script. 😸

• It can now handle read-only files.
• it is somewhat more explicit regarding what it's doing in some circumstances. For example, it now informs the user when no changes are made.
• It now accepts two parameters: -unattended and -log
• Minor improvements here and there.

As you might have guessed, the -unattended parameter skips the parts where user input would normally be required for the script to continue. On the other hand, the -log parameter redirects the script's output to user.js-update-log.txt incrementally. The parameters can be added to a shortcut's path field, or when setting up the script to run as a scheduled task.

I'm quite satisfied with it as it is now, but please make suggestions/requests/observations if you have any.

EDIT: Fixed the new logging feature messing with the script output when disabled 🤦‍♂️

[earthlng]

Very nice changes. I've reviewed + tested it and can confirm it works as advertised :) 👍
As for suggestions/requests/observations maybe add >nul to the powershell command? What do you think? Other than that I can't think of anything else to make this any better, good stuff! Thanks

[claustromaniac]

I finally taught myself to open pull requests and just opened one to change some minor details that I noticed today lol

I enjoy small projects like this once in a while, so it's been my pleasure, really. I'm the one who is grateful for all that you guys do for us here.

[earthlng]

I enjoy small projects like this once in a while

My words exactly and now I'm stuck here xD Anyway, thanks a lot for contributing

[claustromaniac]

Some other potential improvements I've come up with that may be worth considering:

• Remove comments from the resulting user.js file. This would reduce file size to about a 10% of the original size, and could be added as an optional feature with a switch like -stripcomments.
• Open a changelog (or the like) after a successful update. This could also be optional and could happen only if changes are made. The script would take the user to a static URL to read the release notes, or maybe the URL could be inside the user.js file somewhere. Or something like that. There could be other ways to do it.
• Allow merging multiple override files with a common prefix/suffix.
• Make the -unattended switch run the script completely silent (hiding the console, or minimising it). Unfortunately this would require a separate VBS or third-party software. Or building the entire thing from scratch in another language. Or maybe not.. I'll test something later.
• Have the updater auto update itself? xD Probably too much.

[earthlng]

Allow merging multiple override files with a common prefix/suffix.

I like this one. Like fe. (re-)create the user-overrides.js from all the .js files in the preferences folder. That would be neat. The other ideas sound complicated to implement, fe re: changelog - you already end up with a new user.js and the backup and can simply compare the 2 to see all the changes.

[claustromaniac]

(re-)create the user-overrides.js from all the .js files in the preferences folder

Actually, I was thinking more of something like appending all user-something.js files to user.js alphabetically, without modifying any other files, because user.js is the only one Firefox cares about. It would mean that you can have as many separate override files as you want and they will all count.

Maybe a mix of both concepts could be achieved if the script merged all js files in a specific subfolder into a new user-overrides.js file and then appended that one to user.js, but then user-overrides.js would be redundant.

changelog - you already end up with a new user.js and the backup and can simply compare the 2 to see all the changes.

The point of having the script bring up a changelog upon completion would be simply automation, and it would go hand in hand with the option to run it unattended. But you are right. It's probably not worth it.

As for some of my other ideas:

Make the -unattended switch run the script completely silent

Apparently, the task scheduler can hide the console window when both the Run whether user is logged on or not and Hidden options are selected. A great reason to not even bother.

Have the updater auto update itself?

The logic goes like this: Introducing an updater means taking away some of the hassle of updating files, but it isn't all that much beneficial if you now have to manually update the updater. Ideally, one would simply write a functional updater and leave it alone, but you never really know if the future will demand making further changes to it or not.

What I can't make up my mind about is the how(s). I mean, there are many ways to code it, but when should the updater attempt to update itself? Should it be optional? How?

EDIT: I can't help but feel a little bit like this while I think of all this. Just a little bit.

[overdodactyl]

@earthlng:

fix for the profile detection problem: 500c129
It's based on info I found online but I don't have a Mac and can't test it. Let me know if it doesn't work.

I apologize for the delayed response, I've been away from the computer for the past several days. The only issue with using greadlink on a Mac is that it's not a native command. In order for it to work, you have to first install the coreutils package:

brew install coreutils

Does the approach mentioned in my previous comment (cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd) work on a linux system (from reading online, I think it should)? If so, I think this might be a better approach as Mac users wouldn't have to install any extra dependencies.

@Thorin-Oakenpants

Thanks for the invite!!

I need to catch myself up with @claustromaniac - both with improvements to the script and learning to submit pull requests :)

Edit: Here is a link discussing readlink/greadlink on a mac:

http://biercoff.com/fixing-readlink-illegal-option-f-error-on-a-mac/

[earthlng]

@claustromaniac

EDIT: I can't help but feel a little bit like this while I think of all this. Just a little bit.

LOL, so true! I feel like that a lot xD

Maybe a mix of both concepts could be achieved if the script merged all js files in a specific subfolder into a new user-overrides.js file and then appended that one to user.js, but then user-overrides.js would be redundant.

If someone has multiple js overrides I just think it would be nicer to have them all in a subfolder.

@overdodactyl

greadlink on a Mac is that it's not a native command

oh okay, that's not great. I think it needs some sort of readlink command to work in all cases. I don't think your suggested code does that. From what I remember there can be problems if you call it with ./updater.sh and also when using a symlink. They mentioned stat as an alternative I think. Will try to find the link where I read that.

[earthlng]

@overdodactyl can you test if the change in my PR works for you, please? PR: #284

[overdodactyl]

@earthlng I didn't have luck with PR 285, unfortunately. I tried a different approach based off here and had success in all the profiles I tested. I tried submitting my first pull request with the changes: PR 285.

[earthlng]

Your PR adds a function realpath_osx() but calls realpath. (?)
I changed my PR to fallback to what you suggested (cd "$( dirname "${BASH_SOURCE[0]}" )") and merged it. It doesn't work in all cases but it should be good enough for most people.

[overdodactyl]

I was just looking into what you mentioned earlier with stat, I'll let you know if I figure something out. The previous use of just "${BASH_SOURCE[0]}" as the alternative to readlink and greadlink results in user.js being put in the home directory instead of the profile directory.

It doesn't work in all cases

Sounds like it might get a little whacky with the use of symlinks?

Your PR adds a function realpath_osx() but calls realpath. (?)

My mistake. When I was testing it I used realpath, then when I copied it over to github I thought it would be good to change it to realpath_osx to make it more clear what it was used for...forgot to change it in both places.

[claustromaniac]

I need to catch myself up with @claustromaniac - both with improvements to the script and learning to submit pull requests :)

@overdodactyl For your own good, do not try to catch up. It's too late for me, but you can still run away. Do it while you still can, before you begin the quick descent into madness.

[claustromaniac]

I haven't tested the latest version of my script exhaustively, so I might have missed something.

Be sure to let me know if you guys find anything odd, or if you'd like me to explain in more detail how it works. I know the code itself is awful to read, but that was part of the charm. I don't have too much experience with batch scripts, so it was a fun challenge.

[earthlng]

I gave up trying to fix the missing !s and ^s, I just can't get it to work. For user_pref lines in override files there are easy workarounds to keep the lines un-merged, fe by prepending a /* don't merge this */ or simply a TAB. We can mention that in the wiki. Pants and I just have to remember not to use those characters in comments behind user_pref lines.

[claustromaniac]

@earthlng I see a lot has been going on while I wasn't looking heh.

I still haven't given up on that. To be honest, I didn't have time to test what I wanted to test yet.

As for the loopìng auto-update part, it's pretty easy to fix. I'll submit another PR soon (hopefully).

[earthlng]

If you have a user-overrides.js and run the script without any parameters it will simply append your overrides to the end of the user.js. If you instead have multiple override .js files in profile/preferences you can rename that folder to user.js-overrides and run the script with the -multioverrides switch and it will append all those files to the end of the user.js.

And then there's the -merge switch which will do an in-place replacement of active user_pref lines. for example the original user.js has a line

user_pref("whatever", true);

and the user-overrides.js has

user_pref("whatever", false); /* 1234 */

with -merge, the final user.js will have that line replaced with the exact line from the overrides.
-merge can be used with a single user-overrides.js or in combination with -multioverrides for multiple files.

Do they have to be on a line starting with "user_pref" ?

yes. But if someone wants to use the -merge switch and also likes to have comments with !s or ^s, they can either add a TAB or a JS-multi-line comment in front of the user_pref line, fe.

/* don't merge this */ user_pref("whatever", false); // !!! THIS NEEDS TO BE FALSE, WTF !!!

This will keep that line at the bottom of the user.js, together with any other overrides that didn't get merged, for example because a certain pref does not exist or is inactive in the user.js.

user.js.parrot lines are NOT merged and can contain those 2 characters.

All we have to do, is not use ! or ^ in comments behind user_pref lines.

[claustromaniac]

@earthlng Actually, after some testing and reading I found out carets ( ^ ) are unaffected.

The issue with exclamation marks is that they are removed if found anywhere in user_pref lines (except _user.js.parrot lines because those are never merged), and if there are at least two exclamation marks in the same line, everything in between them will get removed as well.

I still haven't had any luck fixing it.

[earthlng]

Yeah I don't think we need to bother with trying to fix it, we'll just note it in the wiki.