bug fixes and improvements

CHANGELOG.md

@@ -1,15 +1,31 @@
 # Changelog
+
 All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [1.1.1] - 2019-01-20
+
+### Added
+
+- Contribution guide and code of conduct
+
+### Fixed
+
+- Bug introduced by updates to crt.sh json that stopped ability to pull data from api
+- Miscellaneous code fixes
+- Updated readme
+
 ## [1.1.0] - 2018-12-27
+
 ### Added
+
 - Changelog
 - Check for new version available
 - Concurrent A record lookups
 - Tabular report
 
 ### Fixed
+
 - Improved error handling

CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at [email protected]. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

CONTRIBUTING.md

@@ -0,0 +1,18 @@
+# Contributing
+
+When contributing to this repository, please first discuss the change you wish to make via issue,
+email, or any other method with the owners of this repository before making a change.
+
+Please note we have a [code of conduct](https://github.com/netevert/delator/blob/master/CODE_OF_CONDUCT.md), please follow it in all your interactions with the project.
+
+## Pull Request Process
+
+1. To submit a successful Pull Request ensure that each new function added is accompanied by related tests to ensure successful build passes and 100% code coverage.
+2. Ensure any install or build dependencies are removed before the end of the layer when doing a
+   build.
+3. Update the README.md and CHANGELOG.md with details of changes to the interface, this includes new environment
+   variables, exposed ports, useful file locations and container parameters.
+4. Increase the version numbers in any examples files, the README.md and CHANGELOG.md to the new version that this
+   Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).
+5. You may merge the Pull Request in once you have the sign-off of the project owner, or if you
+   do not have permission to do that, you may request the project owner to merge it for you.

README.md

@@ -7,14 +7,18 @@
 [![Maintenance](https://img.shields.io/maintenance/yes/2019.svg?style=flat-square)]()
 [![GitHub last commit](https://img.shields.io/github/last-commit/netevert/delator.svg?style=flat-square)](https://github.com/netevert/delator/commit/master)
 
-DELATOR (*lat.* **informer**) is a tool to perform subdomain enumeration and initial reconnaissance through the abusing of certificate transparency logs. It expands on the original work done by [Sheila A. Berta](https://github.com/UnaPibaGeek) with her [CTFR](https://github.com/UnaPibaGeek/ctfr) tool and leverages the speed and power of [Go](https://golang.org/). 
+DELATOR (*lat.* **informer**) is a tool to perform subdomain enumeration and initial reconnaissance through the abusing of certificate transparency logs. It expands on the original work done by [Sheila A. Berta](https://github.com/UnaPibaGeek) with her [CTFR](https://github.com/UnaPibaGeek/ctfr) tool and leverages the speed and power of [Go](https://golang.org/).
 
 ![demo](https://github.com/netevert/delator/blob/master/docs/demo.gif)
 
-Using DELATOR is as simple as running: 
+Using DELATOR is as simple as running:
 
     ./delator -d facebook.com
 
+DELATOR can also be instructed to resolve any subdomains found, giving a first indication of any live sites:
+
+    ./delator -d facebook.com -a
+
 Installation
 ============
 There are two ways to install dnsmorph on your system:
@@ -41,4 +45,6 @@ This project adheres to [Semantic Versioning](https://semver.org/).
 Like it?
 =========
 
- **DELATOR is under active development** so do make sure you check back frequently for new releases. However if you like the tool please consider contributing.
+ **DELATOR is under active development** so make sure you check back frequently for new releases. However if you like the tool please consider contributing.
+
+ A particular issue with DELATOR and [CTFR](https://github.com/UnaPibaGeek/ctfr) is that both tools rely entirely on [Comodo's Certificate Search API](https://crt.sh/). If the API is discontinued the tools will cease to work. A [new version](https://github.com/netevert/delator/tree/v.1.2.0) of DELATOR is currently in the works that will allow for the download of subdomains directly from certificate transparency logs and storage in a local database. This would allow for later scanning for subdomains on demand and give DELATOR the ability to retain it's usefulness should Comodo's API disappear.

delator.go

@@ -13,7 +13,6 @@ import (
 	"net/url"
 	"os"
 	"regexp"
-	"strings"
 	"sync"
 	"text/tabwriter"
 	"time"
@@ -34,25 +33,25 @@ var (
 	ver             = flag.Bool("v", false, "check version")
 	utilDescription = "delator -d domain [-av]"
 	myClient        = &http.Client{Timeout: 10 * time.Second}
-	appVersion      = "1.1.0"
+	appVersion      = "1.1.1"
 	banner          = `
 8"""8 8""" 8    8"""8 ""8"" 8""88 8""8  
 8e  8 8eee 8e   8eee8   8e  8   8 8ee8e
 88  8 88   88   88  8   88  8   8 88  8
 88ee8 88ee 88ee 88  8   88  8eee8 88  8`
 )
 
-type Data struct {
-	Issuer_ca_id        int    `json:"issuer_ca_id"`
-	Issuer_name         string `json:"issuer_name"`
-	Name_value          string `json:"name_value"`
-	Min_cert_id         int    `json:"min_cert_id"`
-	Min_entry_timestamp string `json:"min_entry_timestamp"`
-	Not_after           string `json:"not_after"`
-	Not_before          string `json:"not_before"`
+type data struct {
+	IssuerCaID        int    `json:"issuer_ca_id"`
+	IssuerName         string `json:"issuer_name"`
+	NameValue          string `json:"name_value"`
+	MinCertID         int    `json:"min_cert_id"`
+	MinEntryTimestamp string `json:"min_entry_timestamp"`
+	NotAfter           string `json:"not_after"`
+	NotBefore          string `json:"not_before"`
 }
 
-type Record struct {
+type record struct {
 	Subdomain string `json:"subdomain"`
 	A         string `json:"a_record"`
 }
@@ -64,8 +63,8 @@ func printError(err string) {
 }
 
 // helper function to grab url and robustly handle errors
-func grabUrl(Url string) (resp *http.Response) {
-	resp, err := http.Get(Url)
+func grabURL(URL string) (resp *http.Response) {
+	resp, err := http.Get(URL)
 	if err, ok := err.(*url.Error); ok {
 		if err.Timeout() {
 			printError("request timed out")
@@ -82,39 +81,37 @@ func grabUrl(Url string) (resp *http.Response) {
 }
 
 // fetches certificate transparency json data
-func fetchData(Url string) []Data {
-	res := grabUrl(Url)
+func fetchData(URL string) []data {
+	res := grabURL(URL)
 	body, err := ioutil.ReadAll(res.Body)
 	if err != nil {
 		fmt.Println(err)
 	}
-	result := strings.Replace(string(body), "}{", "},{", -1)
-	d := fmt.Sprintf("[%s]", result)
 
-	keys := make([]Data, 0)
-	json.Unmarshal([]byte(d), &keys)
+	keys := make([]data, 0)
+	json.Unmarshal([]byte(body), &keys)
 	return keys
 }
 
 // deduplicates and prints subdomains
-func printData(data []Data) {
+func printData(Data []data) {
 	counter := make(map[string]int)
-	for _, i := range data {
-		counter[i.Name_value]++
-		if counter[i.Name_value] == 1 {
-			y.Println(i.Name_value)
+	for _, i := range Data {
+		counter[i.NameValue]++
+		if counter[i.NameValue] == 1 {
+			y.Println(i.NameValue)
 		}
 	}
 }
 
 // deduplicates and returns subdomain list
-func extractSubdomains(data []Data) []string {
+func extractSubdomains(Data []data) []string {
 	counter := make(map[string]int)
 	var subdomains []string
-	for _, i := range data {
-		counter[i.Name_value]++
-		if counter[i.Name_value] == 1 {
-			subdomains = append(subdomains, i.Name_value)
+	for _, i := range Data {
+		counter[i.NameValue]++
+		if counter[i.NameValue] == 1 {
+			subdomains = append(subdomains, i.NameValue)
 		}
 	}
 	return subdomains
@@ -169,9 +166,9 @@ func aLookup(subdomain string) string {
 }
 
 // performs lookups on individual subdomain record
-func doLookups(subdomain string, resolve bool, out chan<- Record) {
+func doLookups(subdomain string, resolve bool, out chan<- record) {
 	defer wg.Done()
-	r := new(Record)
+	r := new(record)
 	r.Subdomain = subdomain
 	if resolve {
 		r.A = aLookup(r.Subdomain)
@@ -180,22 +177,22 @@ func doLookups(subdomain string, resolve bool, out chan<- Record) {
 }
 
 // runs bulk lookups on list of subdomains
-func runConcurrentLookups(subdomains []string, resolve bool, out chan<- Record) {
+func runConcurrentLookups(subdomains []string, resolve bool, out chan<- record) {
 	for _, subdomain := range subdomains {
 		wg.Add(1)
 		go doLookups(subdomain, resolve, out)
 	}
 }
 
 // helper function to wait for goroutines collection to finish and close channel
-func monitorWorker(wg *sync.WaitGroup, channel chan Record) {
+func monitorWorker(wg *sync.WaitGroup, channel chan record) {
 	wg.Wait()
 	close(channel)
 }
 
 // helper function to run lookups and print results
 func printResults(subdomains []string) {
-	out := make(chan Record)
+	out := make(chan record)
 	writer.Init(os.Stdout, 14, 8, 0, '\t', tabwriter.DiscardEmptyColumns)
 	runConcurrentLookups(subdomains, *resolve, out)
 	go monitorWorker(wg, out)