older versions of spfx incompatible with corresponding docker image, and other gulp related issues

[flowerbot]

Let me tell you a story.

I started a new spFx project in docker following directions here: https://www.donkirkham.com/blog/spfx-docker. Excellent and simple for me, since I'm new to docker. I followed the directions for v 1.14, certificate is nicely generated automatically, I download and trust it from its easy to find location, and everything worked beautifully EXCEPT the changes I was making weren't being served. I could only see them by killing the server and doing another 'gulp serve'. Dr Google had no useful help for me, I don't know how to fix it to make it work and don't want to waste time down that rabbit hole ....

So I thought ok, I know reload works in earlier versions (had played around with PnP samples which were all using version 1.10), so I'll try a docker image for an earlier version of spFx. Set it all up the same way, starting with the scaffold for v 1.10, EXCEPT gulp version in the scaffold is the latest version of gulp which is apparently incompatible with earlier spFx versions, and incompatible with the container image. Has the (unnecessary and code-breaking?) update to the scaffold package list for old versions ever been tested with the docker image? I think not.

Sure enough when I go back to compare to my functioning v 1.10 PnP sample, its using an earlier version, so I go to look for information on incompatibilities, and I discover there are significant breaking changes between version 3* and 4* of gulp, and I've got to ask, is anyone on the team even aware of this problem? Why would gulp be updated in older versions of the scaffold without appropriate code changes to make it function correctly? Why would gulp be updated in the latest version without appropriate code changes to make it function correctly? In the docker-world at least, spFx doesn't seem to work as required according to the scaffold now, for any version. At least not from v 1.10 onwards.

I think what I'm going to do next is try another v 1.10 container, but use the package list from a PnP sample, so I should get the correct gulp version. Wish me luck.

[AJIXuMuK]

@flowerbot - I'm really sorry for all the challenges you're experiencing.
To be clear in expectations: SPFx product team is not responsible for Docker images and its support is not our top priority.
We have a compatibility page that also has links to the release notes. And in the release notes you can see when the updates to Gulp version were done.
Also, I would strongly recommend using nvs as you may have different versions of Node and installed modules in that case.

[DonKirkham]

@flowerbot Thanks for bringing this to our attention. I will attempt to replicate it to better understand what you are experiencing. As Alex (@AJIXuMuK) stated, Microsoft does not build the Docker images. Those are done by the Microsoft 365 Platform Community (PnP). I will tag a couple of people that work with those images, as they may have more insight. @hugoabernier @waldekmastykarz

[hugoabernier]

@flowerbot, I realize how frustrating it must be. I would suggest that you open an issue in the github.com/pnp/sp-dev-fx-webparts repository for the specific sample that does not work so we can look into it.

[flowerbot]

Thanks for the response and comments.

@AJIXuMuK ,I wrote my story here about my experience so far with spFx Docker images because this is where the Docker image page for spFx directed me. As to nvm, have used that in the past but in a docker container, nvm should be unnecessary? nvm would be more relevant if I was developing on my local machine, but I haven't tried to work with spFx on my local machine since I discovered Docker, because the idea of developing inside a Docker container looked so much like 'the answer' to the usual avalanche of spFx incompatibility issues, that I'm determined to get spFx docker images working as expected and I would hope someone at MS is too. A non-docker question for you as follow-up though, does the current (1.14) version of spFx reload changes while developing? Because the version in the docker image doesn't.

@hugoabernier your PnP webpart sample works, all of the other ones I've tried mostly work too. If there are minor incompatibilities, they have so far been easily fixed. I built my first Dockerised web part by using a copy of yours as a starting point: with that container, the 'reload' function works, sort of - had to refresh my pages with cache disabled multiple times to see changes, not wonderful, but tolerable. Its the current Docker images for spFx (1.10-1.14) that don't work for me. The 1.14 image doesn't load changes, and with the 1.10/11 images, gulp is broken.

@DonKirkham Without your guide I would not have gotten to this point. Any light you can bring to help me solve my current problem will also be greatly appreciated.

I hope I can find a way through, still haven't though. There were more differences than just package versions between the functioning v1.10 PnP sample and the the current non-functioning v 1.10 image. I then went to trying figure out how to global install anything in a container from an spFx Docker image. There's no 'root user', and no other user I can find authorised as such, and the spfx user ('spfx') which (I think) did the initial install from Docker isn't authorised to now global install anything extra (eg to fix the gulp issue). I've read that node installs packages as some other user who is later not authorised by Docker, but I don't fully understand that yet. I admit my Docker and Linux knowledge is very low at present, and I'm running through a 3 hour video Docker tutorial to see if there's anything in there that will help me - eg can I build my own spFx image from scratch with all the stuff I want, including globally installed things like the version of Gulp that's going to work with the version of spFx I want to use. I'm sure its possible but can "I" do it? I'd rather not, and was thinking the pre-made Docker images would mean I didn't have to. Part-way through my crash course in Docker, I'm thinking I might go take a peak at the spFx image Dockerfile and see how much I understand/can fix to make my own now .. ok can't do that :P And so it goes.

But really I just want to get back to my project. At this stage I'd happily use a proven PnP sample and convert that to my project, if only it was easier to rename things ....

Again, appreciate all your responses so far.

[AJIXuMuK]

Yes, reload is working for the 1.14.
Just to be completely clear - the files will be updated and you will see that in command prompt/terminal. But you need to refresh your page in browser manually as you're using debug query string.

[DonKirkham]

I am not 100% sure what issue we are trying to address, so let me tackle what I see and try to explain some of the "quirks" with Docker images:

• @AJIXuMuK is 100% correct about the automatic reload. In my experience, the automatic reload only worked in the local workbench (when running gulp serve in the local machine.) The hosted workbench and using the query string method to put webpart on an actual page has always required me to hit refresh once the project is updated. (Once the "Finished subtask 'reload'" appears in the terminal, you can refresh the page in the browser.) Is that what you mean by "reload not working?"

• My understanding Docker Images is that the image is locked, so you can't make changes to the global packages. I am not sure if there is a workaround. The intent for including these in the samples was to allow a user to quickly get a sample running, since the correct environment was included in the project via the Docker image, so modifying the environment is not needed.

• The images used in the samples repo all have gulp installed in the global package space. The version of gulp installed is the same verion that will be in the package.json file. This should always be compatible for the SPFx version of the SharePoint Yeoman generator. If you want to use a different SPFX version, you should use the image that matches the SPFx version.

• Based on recent guidance changes, I never install gulp in the global space, but install gulp-cli instead. This prevents gulp incompatibilites if the SPFx version changes, but that should not be an issue with the Docker images used in the samples repo based on the last bullet above

• You are correct that nvm is not needed in the Docker images included with the samples. This is because each image has everything included. If you build a custom Docker image, which I discuss below, or are developing on a local machine where you might need to change Node versions based on the version of SPFx, nvm (or nvs) is critical to easily change between Node versions. With nvm, each version of Node that you install comes with a clean environment, so you can install any packages you need. My recommendation is that you install a Node version that is unique for a single version of SPFx. Even if you need several versions of SPFx that have the same package dependencies, just install at new version of Node. e.g., Node v10 works with SPFx 1.10.x and 1.11.x, so install Node v10.0 for SPFx 1.10.x and Node v10.1 for SPFx 1.11.x. Each version of Node has a unique environment for the SPFx version, even if those environments are identical.

• Before I started using prebuilt Docker images, I used Dockerfiles to custom build my images. (I should add this to my blog.) This works by taking a generic base image for Node and adding whatever you want. You can change any of the packages and load custom tools, like the M365 CLI tool. I stopped doing this because the prebuild images were simpler for just working on a single project. If I am upgrading a sample to a new version or need something else that the prebuilt image doesn't allow, I just use my local machine environment (and nvm) to do that work.

Bottom line is that Docker can be useful in many cases, but it has it's limits.

If I am still missing the issue you are trying to raise, please let me know and we'll work through it!

PS. I am very excited to know someone read my blog and used information in it!!