Shog build system

An initial version of shog build system that uses Ninja as a build
backend. Shog allows to describes targets/files and dependencies in Ruby
langauge.

CHANGES.md

@@ -0,0 +1,3 @@
+# v0.1.0 (2018-01-08)
+
+- Initial version.

CODE_OF_CONDUCT.md

@@ -0,0 +1,61 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making
+participation in our project and our community a harassment-free experience for everyone, regardless of age, body size,
+disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race,
+religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take
+appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits,
+issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any
+contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the
+project or its community. Examples of representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed representative at an online or offline
+event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at
+[[email protected]](mailto:[email protected]). All complaints will be reviewed and
+investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project
+team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific
+enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent
+repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

CONTRIBUTING.md

@@ -0,0 +1,22 @@
+# Contributing
+
+Thanks for taking an interest in this open source project. Your support and involvement is greatly
+appreciated. The following sections detail what you need to know in order to contribute.
+
+## Code
+
+0. Read the project README before starting.
+0. Fork the `master` branch of this repository and clone the fork locally.
+0. Ensure there are no setup, usage, and/or test issues.
+0. Add tests for new functionality and ensure they pass.
+0. Submit a pull request, follow the instructions it provides, and ensure the build passes.
+
+## Issues
+
+0. Submit an issue via the GitHub Issues tab (assuming one does not
+   already exist) and follow the instructions it provides.
+
+## Feedback
+
+- Expect a response within one to three business days.
+- Changes, alternatives, and/or improvements might be suggested upon review.

Gemfile

@@ -0,0 +1,2 @@
+source "https://rubygems.org"
+gemspec

LICENSE.md

@@ -0,0 +1,20 @@
+Copyright (c) 2017 [Anatol Pomozov](https://github.com/anatol).
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -0,0 +1,7 @@
+# Shog Build Generator
+
+A Ruby generator for [Ninja](https://ninja-build.org/) build system.
+
+## Credits
+
+Developed by Anatol Pomozov.

bin/shog

@@ -0,0 +1,5 @@
+#!/usr/bin/ruby
+
+require_relative '../lib/runner'
+
+Shog::Runner.run(ARGV)

lib/context.rb

@@ -0,0 +1,87 @@
+require_relative 'util'
+require_relative 'pathset'
+
+module Shog
+  # An instance of this class is visible to our build scripts with name @shog
+  class Context
+    attr_accessor :default_target, :rule, :config
+
+    def initialize(backend, emitter)
+      @backend = backend
+      @emitter = emitter
+
+      @rule = {}
+      @default_target = PathSet.new
+      @config = {}
+    end
+
+    def register_rule(type)
+      r = type.new
+      @emitter.rule(r)
+      @rule[r.id] = r
+    end
+
+    def bind
+      binding
+    end
+
+    def visit_dir(dir, new_ctx = true)
+      old_pwd = Path.pwd
+      ctx = new_ctx ? deep_clone() : self
+
+      Path.pwd = File.join(Path.pwd, dir)
+      script = cwd('shog.build')
+      @rule[:generate_build].deps << script
+      script_name = script.path
+      build_script = File.read(script_name)
+      out = ctx.bind.eval(build_script, script_name)
+
+      Path.pwd = old_pwd
+
+      return out
+    end
+
+    def visit(dirs)
+      case dirs
+      when String then visit_dir(dirs)
+      when Array then dirs.map { |d| visit_dir(d) }.flatten
+      else raise "Unknown object type for dirs: #{dirs.class.name}"
+      end
+    end
+
+    def cwd(src)
+      Path.make(src)
+    end
+
+    def emit(rule_id, src, params = {})
+      r = @rule[rule_id]
+      raise "Rule #{rule_id} is not registered" unless r
+
+      params[:input] = PathSet.make(src)
+      target = r.target(params)
+      # fix path to inputs, outputs, includes
+
+      @emitter.emit(target)
+      return target[:output]
+    end
+
+    def emit_each(rule_id, srcs, params = {})
+      out = []
+      for s in srcs
+        p = params.dup
+        p[:input] = s
+        out += emit(rule_id, s, p)
+      end
+      return out
+    end
+
+    def deep_clone
+      ctx = Context.new(@backend, @emitter)
+      ctx.rule = @rule.deep_clone
+      ctx.rule[:generate_build].deps = @rule[:generate_build].deps # deps are global across the build
+      ctx.default_target = @default_target.deep_clone
+      ctx.config = @config
+      ctx
+    end
+  end
+end

lib/generator.rb

@@ -0,0 +1,39 @@
+require_relative 'context'
+require_relative 'rule/cc'
+require_relative 'rule/link'
+require_relative 'rule/objcopy'
+require_relative 'rule/kconfig'
+require_relative 'rule/generate_build'
+require_relative 'rule/yacc'
+
+module Shog
+  class Generator
+    def initialize(backend)
+      @backend = backend
+    end
+
+    def generate
+      emitter = @backend.emitter
+      ctx = Context.new(@backend, emitter)
+
+      if File.exists?('Kconfig') and not File.exists?('.config')
+        system 'conf --alldefconfig -s Kconfig'
+      end
+
+      # Register all rules
+      ctx.register_rule(CC)
+      ctx.register_rule(Link)
+      ctx.register_rule(ObjCopy)
+      ctx.register_rule(Kconfig)
+      ctx.register_rule(GenerateBuild)
+      ctx.register_rule(Yacc)
+
+      Path.pwd = '.'
+      ctx.visit_dir('.', false)
+
+      emitter.default(ctx.default_target)
+
+      emitter.finish
+    end
+  end
+end

lib/ninja.rb

@@ -0,0 +1,65 @@
+module Shog
+  class Ninja
+    attr_reader :backend_file
+
+    def initialize
+      @out_dir = 'out'
+      @backend_file = File.join(@out_dir, 'build.ninja')
+    end
+
+    def configured?
+      File.exists?(@backend_file)
+    end
+
+    def emitter
+      Emitter.new(@backend_file)
+    end
+
+    def run
+      system "ninja -C #{@out_dir}"
+    end
+
+    class Emitter
+      def initialize(file)
+        @out = File.open(file, 'w')
+      end
+
+      def finish
+        @out.close
+      end
+
+      def default(target)
+        unless target.empty?
+          @out.puts "default #{target.join(' ')}"
+          @out.puts
+        end
+      end
+
+      def rule(r)
+        vars = r.rule
+        @out.puts "rule #{r.id.to_s}"
+        for k, v in vars
+          @out.puts "  #{k}=#{v}"
+        end
+        @out.puts
+      end
+
+      def emit(target)
+        rule = target[:rule]
+        input = target[:input].join(' ')
+        implicit_input = if target[:implicit_input] and not target[:implicit_input].empty?
+                           ' | ' + target[:implicit_input].join(' ')
+                         else
+                           ''
+                         end
+        output = target[:output].join(' ')
+        variables = target[:variables]
+        @out.puts "build #{output}: #{rule} #{input}#{implicit_input}"
+        for k, v in variables
+          @out.puts "  #{k}=#{v}"
+        end if variables
+        @out.puts
+      end
+    end
+  end
+end

lib/path.rb

@@ -0,0 +1,70 @@
+require 'pathname'
+
+module Shog
+  class Path
+    @pwd = '.'
+
+    class << self
+      attr_accessor :pwd
+    end
+
+    attr_reader :path, :outoftree, :absolute
+
+    private :initialize
+
+    def initialize(path, outoftree, absolute = false)
+      @path = path
+      @outoftree = outoftree
+      @absolute = absolute
+    end
+
+    def self.normalize(path)
+      Pathname.new(path).cleanpath.to_s
+    end
+
+    def self.make(path, params = {})
+      if path.is_a?(String)
+        abolute = false
+        if params.key?(:absolute) and params[:absolute]
+          path = File.expand_path(path)
+          absolute = true
+        elsif not params.key?(:root) and not params[:root]
+          path = File.join(Path.pwd, path)
+        end
+        path = Path.normalize(path)
+        outoftree = if params.key?(:outoftree)
+                      params[:outoftree]
+                    else
+                      false # by default we assume input file
+                    end
+        return Path.new(path, outoftree, absolute)
+      elsif path.is_a?(Path)
+        if params.key?(:outoftree) and params[:outoftree] != path.outoftree
+          return Path.new(path.path, params[:outoftree])
+        else
+          return path
+        end
+      else
+        raise "Cannot make path from #{path}"
+      end
+    end
+
+    def dir
+      path = File.dirname(@path)
+      Path.new(path, @outoftree)
+    end
+
+    def with_suffix(suffix)
+      Path.new(Path.normalize(@path + suffix), @outoftree)
+    end
+
+    def to_str
+      # as we build files by ninja from inside outoftree directory then outs should not have any prefixes, but inputs should be accessed by '../'
+      if @outoftree or @absolute
+        @path
+      else
+        File.join('..', @path)
+      end
+    end
+  end
+end