Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to AWS v3 library #569

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Move to AWS v3 library #569

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
95c64e2
aws v3 support
jjlimepoint Aug 21, 2018
43d51f7
Merge branch 'master' into aws-v3
tas50 Oct 1, 2018
f0e766b
fixup AWS gem deps to be ~> 1.0 as per code review
jjlimepoint Oct 1, 2018
e43b619
add a missing require to aws_key_pair
jjlimepoint Oct 1, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ env:
- RAKE_TASK=spec
# We run the tests in different environments so they don't try to overwrite each other (deleting, creating the same VPC)
# If we need to we can get rid of this but its nice for speed
- RAKE_TASK="travis[integration]" AWS_TRAVIS_DRIVER=aws::us-west-2
- RAKE_TASK="travis[super_slow]" AWS_TRAVIS_DRIVER=aws::us-west-1
- RAKE_TASK="travis[integration]" AWS_TEST_DRIVER=aws::us-west-2
- RAKE_TASK="travis[super_slow]" AWS_TEST_DRIVER=aws::us-west-1
# machine_image is a special snowflake - they take so long to run we need to give them their own builder
- RAKE_TASK="machine_image" AWS_TRAVIS_DRIVER=aws::eu-west-1
- RAKE_TASK="machine_image" AWS_TEST_DRIVER=aws::eu-west-1

bundler_args: --jobs 7 --without docs debug

Expand Down
17 changes: 16 additions & 1 deletion chef-provisioning-aws.gemspec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,22 @@ Gem::Specification.new do |s|

s.add_dependency "chef-provisioning", ">= 1.0", "< 3.0"

s.add_dependency "aws-sdk", [">= 2.2.18", "< 3.0"]
# all currently supported AWS services
s.add_dependency "aws-sdk-core", [">= 3.0", "< 4.0"]
s.add_dependency "aws-sdk-ec2", [">= 1.42.0", "< 2.0"]
s.add_dependency "aws-sdk-s3", [">= 1.17.0", "< 2.0"]
s.add_dependency "aws-sdk-rds", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-route53", [">= 1.0", "< 4.0"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resource deps should all be ~> 1.0 since Amazon is on 1.x right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - thanks for that!

s.add_dependency "aws-sdk-autoscaling", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-cloudwatch", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-cloudsearch", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-cloudsearchdomain", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-elasticache", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-iam", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-opsworks", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-sns", [">= 1.0", "< 4.0"]
s.add_dependency "aws-sdk-sqs", [">= 1.0", "< 4.0"]

s.add_dependency "retryable", "~> 2.0", ">= 2.0.1"
s.add_dependency "ubuntu_ami", "~> 0.4", ">= 0.4.1"

Expand Down
1 change: 0 additions & 1 deletion lib/chef/provider/aws_key_pair.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
require "chef/provider/lwrp_base"
require "chef/provisioning/aws_driver/aws_provider"
require "aws-sdk"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be require "aws-sdk-ec2"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the time I actually removed this deliberatly, since the class doesn't actually use anything from aws-sdk-* directly at all, but rather the things provided by aws_provider. Still, not real harm in specifying it.


class Chef::Provider::AwsKeyPair < Chef::Provisioning::AWSDriver::AWSProvider
provides :aws_key_pair
Expand Down
2 changes: 1 addition & 1 deletion lib/chef/provider/aws_nat_gateway.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# require 'chef/provisioning/aws_driver/aws_provider'
# require "chef/provisioning/aws_driver/aws_provider"
require "retryable"

class Chef::Provider::AwsNatGateway < Chef::Provisioning::AWSDriver::AWSProvider
Expand Down
16 changes: 15 additions & 1 deletion lib/chef/provisioning/aws_driver/aws_resource.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,18 @@
require "aws-sdk"
# Parts of AWS we support - for now this is scattershot, later this will get cleaned up
require "aws-sdk-s3"
require "aws-sdk-ec2"
require "aws-sdk-rds"
require "aws-sdk-route53"
require "aws-sdk-autoscaling"
require "aws-sdk-cloudwatch"
require "aws-sdk-cloudsearch"
require "aws-sdk-cloudsearchdomain"
require "aws-sdk-elasticache"
require "aws-sdk-iam"
require "aws-sdk-opsworks"
require "aws-sdk-sns"
require "aws-sdk-sqs"

require "chef/provisioning/aws_driver/super_lwrp"
require "chef/provisioning/chef_managed_entry_store"
# Enough resources will eventually require this that we put 1 require in here
Expand Down
1 change: 0 additions & 1 deletion lib/chef/provisioning/aws_driver/credentials2.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
require "aws-sdk"
require "aws-sdk-core/credentials"
require "aws-sdk-core/shared_credentials"
require "aws-sdk-core/instance_profile_credentials"
Expand Down
18 changes: 15 additions & 3 deletions lib/chef/provisioning/aws_driver/driver.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,26 @@
require "chef/provisioning/aws_driver/aws_tagger"

require "yaml"
require "aws-sdk"
require "retryable"
require "ubuntu_ami"
require "base64"

# Parts of AWS we support - for now this is scattershot, later this will get cleaned up
require "aws-sdk-s3"
require "aws-sdk-ec2"
require "aws-sdk-rds"
require "aws-sdk-route53"
require "aws-sdk-autoscaling"
require "aws-sdk-cloudwatch"
require "aws-sdk-cloudsearch"
require "aws-sdk-cloudsearchdomain"
require "aws-sdk-elasticache"
require "aws-sdk-iam"
require "aws-sdk-opsworks"
require "aws-sdk-sns"
require "aws-sdk-sqs"

# loads the entire aws-sdk
Aws.eager_autoload!
AWS_V2_SERVICES = {
"EC2" => "ec2",
"Route53" => "route53",
Expand All @@ -38,7 +51,6 @@
"CloudWatch" => "cloudwatch",
"AutoScaling" => "auto_scaling"
}.freeze
Aws.eager_autoload!(services: AWS_V2_SERVICES.keys)

# Need to load the resources after the SDK because `aws_sdk_types` can mess
# up AWS loading if they are loaded too early
Expand Down
2 changes: 1 addition & 1 deletion lib/chef/provisioning/aws_driver/version.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
class Chef
module Provisioning
module AWSDriver
VERSION = "3.0.5".freeze
VERSION = "4.0.0"
end
end
end
2 changes: 1 addition & 1 deletion lib/chef/resource/aws_nat_gateway.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# We provide this class because the AWS SDK V2 does not provide it (as of
# May 2016). We copied the pattern in their SDK so when they do add a real
# resource there shouldn't be a need for much translation.
class Aws::EC2::NatGateway < ::Aws::Resources::Resource
class Aws::EC2::NatGateway # < ::Aws::Resources::Resource
attr_reader :resource, :id, :nat_gateway_id, :vpc_id, :subnet_id, :nat_gateway_addresses

def initialize(id, options = {})
Expand Down
16 changes: 15 additions & 1 deletion spec/aws_support.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,21 @@ def self.extended(other)
require "aws_support/aws_resource_run_wrapper"

# Add AWS to the list of objects which can be matched against a Hash or Array
require "aws-sdk"
# Parts of AWS we support - for now this is scattershot, later this will get cleaned up
require "aws-sdk-s3"
require "aws-sdk-ec2"
require "aws-sdk-rds"
require "aws-sdk-route53"
require "aws-sdk-autoscaling"
require "aws-sdk-cloudwatch"
require "aws-sdk-cloudsearch"
require "aws-sdk-cloudsearchdomain"
require "aws-sdk-elasticache"
require "aws-sdk-iam"
require "aws-sdk-opsworks"
require "aws-sdk-sns"
require "aws-sdk-sqs"

require "aws_support/deep_matcher/matchable_object"
require "aws_support/deep_matcher/matchable_array"
DeepMatcher::MatchableObject.matchable_classes << proc { |o| o.class.name =~ /^(AWS|Aws)::(AutoScaling|EC2|ELB|IAM|S3|RDS|CloudSearch|CloudWatch|Route53|ElasticsearchService)($|::)/ }
Expand Down
2 changes: 1 addition & 1 deletion spec/integration/aws_rds_subnet_group_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
require "spec_helper"
require "aws-sdk"
require "aws-sdk-rds"
require "set"

describe Chef::Resource::AwsRdsSubnetGroup do
Expand Down
2 changes: 1 addition & 1 deletion tools/purge_zone.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

require "json"
require "pp"
require "aws-sdk"
require "aws-sdk-route53"

# it happens that you end up with a HostedZone with no Chef Server entry, and destroying it manually is
# tedious, because you have to delete all the ResourceRecordSets. this script will handle that for you.
Expand Down