Generalizes to support multiple migrators #16

jathak · 2019-04-01T18:29:55Z

Resolves #19.

There's now a Migrator class that additional migrators can extend.

Most migrators will want to extend MigratorBase, which handles the tree traversal. For now, MigratorBase only operates on a single stylesheet, with the dependency migration implemented only by ModuleMigrator.

There's now a `Migrator` class that additional migrators can extend. Most migrators will want to extend `MigratorBase`, which handles the tree traversal. For now, `MigratorBase` only operates on a single stylesheet, with the dependency migration implemented only by `ModuleMigrator`.

nex3

Are you planning to have Migrator stay non-recursive? If so, what's the plan for migrating entire projects at once?

bin/sass_migrator.dart

lib/src/migrator_base.dart

lib/src/migrators/module.dart

test/migrator_test.dart

--migrate-deps is now a global flag, so additional migrators should not need to re-implement dependency migration unless they need additional special logic like the module migrator needs. The generic migration classes are now split into Migrator and MigrationVisitor. Each Migrator is a Command and should contain state for the entire migration run. Most migrators will also want to create a private subclass of MigrationVisitor that they call from Migrator.migrateFile. A new instance of the MigrationVisitor will be created for each stylesheet being migrated, so it contains per-stylesheet state, similar to what StylesheetMigration used to be for the module migrator.

jathak · 2019-04-05T00:35:47Z

I've updated this to now have the generic migration support split into Migrator, which extends Command and contains state for the entire migration run, and MigrationVisitor, which migrators subclass and use to traverse the AST of a single stylesheet.

I've also made the --migrate-deps flag global. The module migrator has some special logic here since it needs to visit all dependencies even if they're not being migrated, but any migrators that don't rely on cross-stylesheet information should be able to support migrating dependencies without any additional code.

nex3

This is shaping up really nicely! Thanks for bearing with my intense reviews 😅.

lib/runner.dart

lib/src/migration_visitor.dart

lib/src/migrators/module.dart

lib/src/utils.dart

pubspec.yaml

The migrator now uses URLs instead of paths, and loads files using the FilesystemImporter. `Migrator.migrated` has been removed in favor of having `Migrator.migrateFile` and `MigrationVisitor.run` return it.

lib/runner.dart

lib/src/migration_visitor.dart

nex3 · 2019-04-09T01:17:32Z

lib/src/migration_visitor.dart

+    visitStylesheet(stylesheet);
+    var results = getMigratedContents();
+    if (results != null) {
+      migrator.migrated[path] = results;


That's a good question. I'd say make the migrated field private to MigrationVisitor and give it total responsibility for handling the output, but you need to pass it through newInstance() somehow...

What if we restructure this so that you don't need to construct multiple visitors at all? Rather than passing a URL to the constructor, pass it to run(). That also means that when we get around to de-duplicating repeated imports/uses (which will probably be necessary for large-scale performance), there's a natural place for that as well.

This does mean that per-file state can't just be stored as final fields on the subclass. But you can override visitStylesheet() to set it temporarily:

@override void visitStylesheet(StylesheetNode node) { var oldNamespaces = _namespaces; var oldAdditionalUseRules = _additionalUseRules; var oldConfigurableVariables = _configurableVariables; _namespaces = {}; _additionalUseRules = Set(); _configurableVariables = normalizedSet(); super.visitStylesheet(node); _namespaces = oldNamespaces; _additionalUseRules = oldAdditionalUseRules; _configurableVariables = oldConfigurableVariables; }

This is basically how Dart Sass's EvaluateVisitor works. WDYT?

lib/src/migration_visitor.dart

lib/src/utils.dart

test/migrator_utils.dart

jathak · 2019-04-12T22:56:24Z

Changed MigrationVisitor to reuse a single instance for dependencies.

Previously, the module migrator needed to access the _configurableVariables of its dependencies' visitors. In the process of changing this, I found #28 and (mostly) fixed this in my changes (though it doesn't yet remove an unnecessary use).

nex3

Would it be possible to factor out the configurable-variables change into a separate PR? It's kind of tough to review this when it's changing multiple things at once.

lib/src/migration_visitor.dart

A single MigrationVisitor is now reused instead of constructing a new instance for each dependency. This also removes support for configurable variables, since the previous implementation of this was both hard to implement with a single MigrationVisitor and incorrect when the configurable variables were declared in an indirect dependency. A subsequent PR will add this back with a new implementation that fixes the bug with indirect dependencies.

jathak · 2019-04-17T23:12:16Z

The old behavior would be hard to implement with a single MigrationVisitor, so I removed support for configurable variables entirely and will add them back with the new implementation in a separate PR.

nex3 · 2019-04-17T23:58:47Z

lib/src/migrators/module.dart

+  final _globalFunctions = normalizedMap<FunctionRule>();
+
+  /// Namespaces of modules used in this stylesheet.
+  Map<Uri, String> _namespaces = {};


I think you can leave off the initializer here; this value will never be used, since it's always going to be set in visitStylesheet().

nex3 · 2019-04-18T00:00:25Z

lib/src/migrators/module.dart

+    if (results == null) return null;
+    var semicolon = _currentUrl.path.endsWith('.sass') ? "" : ";";
+    var uses = _additionalUseRules.map((use) => '@use "$use"$semicolon\n');
+    return uses.join("") + results;


Suggested change

return uses.join("") + results;

return uses.join() + results;

nex3 · 2019-04-18T00:00:45Z

lib/src/migrators/module.dart

+    _localScope = _localScope.parent;
+  }
+
+  /// Adds a namespace to any function call that require it.


Suggested change

/// Adds a namespace to any function call that require it.

/// Adds a namespace to any function call that requires it.

lib/src/utils.dart

jathak mentioned this pull request Apr 1, 2019

Initial implementation of a division migrator #17

Merged

jathak requested a review from nex3 April 1, 2019 18:33

nex3 requested changes Apr 2, 2019

View reviewed changes

jathak added 3 commits April 4, 2019 16:41

Split migrator tests into separate test files

5a58296

Minor formatting fixes

28c09b8

jathak requested a review from nex3 April 5, 2019 00:35

nex3 requested changes Apr 5, 2019

View reviewed changes

Iterate in response to review.

0b6a061

The migrator now uses URLs instead of paths, and loads files using the FilesystemImporter. `Migrator.migrated` has been removed in favor of having `Migrator.migrateFile` and `MigrationVisitor.run` return it.

nex3 reviewed Apr 9, 2019

View reviewed changes

jathak requested a review from nex3 April 12, 2019 22:56

nex3 reviewed Apr 17, 2019

View reviewed changes

lib/src/migration_visitor.dart Outdated Show resolved Hide resolved

lib/src/migration_visitor.dart Outdated Show resolved Hide resolved

jathak force-pushed the generalization branch from 204bc24 to 8bd5417 Compare April 17, 2019 23:10

jathak requested a review from nex3 April 17, 2019 23:12

nex3 approved these changes Apr 18, 2019

View reviewed changes

Fix nits

5ac0dc6

jathak merged commit 29e9ffb into master Apr 18, 2019

jathak deleted the generalization branch April 19, 2019 22:11

nex3 mentioned this pull request Apr 30, 2019

Migrate /-as-division to the divide() function #37

Closed

4 tasks

jathak added module system Part of the module system migrator and removed module system Part of the module system migrator labels Sep 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalizes to support multiple migrators #16

Generalizes to support multiple migrators #16

jathak commented Apr 1, 2019 •

edited

Loading

nex3 left a comment

jathak commented Apr 5, 2019

nex3 left a comment

nex3 Apr 9, 2019

jathak commented Apr 12, 2019

nex3 left a comment

jathak commented Apr 17, 2019

nex3 Apr 17, 2019

nex3 Apr 18, 2019

nex3 Apr 18, 2019

	return uses.join("") + results;
	return uses.join() + results;

	/// Adds a namespace to any function call that require it.
	/// Adds a namespace to any function call that requires it.

Generalizes to support multiple migrators #16

Generalizes to support multiple migrators #16

Conversation

jathak commented Apr 1, 2019 • edited Loading

nex3 left a comment

Choose a reason for hiding this comment

jathak commented Apr 5, 2019

nex3 left a comment

Choose a reason for hiding this comment

nex3 Apr 9, 2019

Choose a reason for hiding this comment

jathak commented Apr 12, 2019

nex3 left a comment

Choose a reason for hiding this comment

jathak commented Apr 17, 2019

nex3 Apr 17, 2019

Choose a reason for hiding this comment

nex3 Apr 18, 2019

Choose a reason for hiding this comment

nex3 Apr 18, 2019

Choose a reason for hiding this comment

jathak commented Apr 1, 2019 •

edited

Loading