-
Notifications
You must be signed in to change notification settings - Fork 0
/
python-list-comprehension-and-sets.html
171 lines (152 loc) · 15.7 KB
/
python-list-comprehension-and-sets.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>Python List Comprehension and Sets</title>
<link rel="stylesheet" href="http://bryanprice.net/theme/css/custom.css" />
<link href="http://bryanprice.net/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bryan Price Atom Feed" />
<!--[if IE]>
<script src="http://html5shiv.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
</head>
<body id="index" class="home">
<header id="banner" class="body">
<h1><a href="http://bryanprice.net/">Bryan Price </a></h1>
<nav><ul>
<li><a href="http://bryanprice.net/pages/about.html">About</a></li>
<li><a href="http://bryanprice.net/pages/coding-projects.html">Coding Projects</a></li>
</ul>
</nav>
<div id="submenu">
<ul>
<li class="active"><a href="http://bryanprice.net/category/blog.html">Blog</a></li>
<li><a href="http://bryanprice.net/category/project-updates.html">Project Updates</a></li>
</ul>
<div>
</header><!-- /#banner -->
<section id="content" class="body">
<article>
<header>
<h1 class="entry-title">
<a href="http://bryanprice.net/python-list-comprehension-and-sets.html" rel="bookmark"
title="Permalink to Python List Comprehension and Sets">Python List Comprehension and Sets</a></h1>
</header>
<div class="entry-content">
<footer class="post-info">
<span>Thu 29 January 2015</span>
<span>| in <a href="http://bryanprice.net/category/blog.html">Blog</a></span>
<span>| tags: <a href="http://bryanprice.net/tag/python.html">Python</a></span>
</footer><!-- /.post-info --> <p>I work for a company that does GIS work, and over the past week or so
I've been working on a script that does a lot of different things. One
of it's subroutines deletes data from Oracle.</p>
<p>First I was told (or I interpreted) it was safe to delete everything
from the Oracle connection. So this is similar to my first stab at the
code that deletes data from Oracle.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">arcpy</span>
<span class="n">arcpy</span><span class="o">.</span><span class="n">env</span><span class="o">.</span><span class="n">workspace</span> <span class="o">=</span> <span class="sa">r</span><span class="s2">"c:\oracle_connection.sde"</span> <span class="c1"># file that holds all the connection info</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListTables</span><span class="p">()</span> <span class="o">+</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListFeatureClasses</span><span class="p">()</span> <span class="o">+</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListDatasets</span><span class="p">():</span>
<span class="n">arcpy</span><span class="o">.</span><span class="n">Delete_management</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
</pre></div>
<p>One of the neat things about python is the ease at which lists can be
pulled together. Simply use the + operator!</p>
<p>The code above lists all tables, feature classes, and datasets. If
you're unfamiliar with ArcGIS, a feature class is simply a table that
has a shape related to each row. A dataset is a container for feature
classes.</p>
<p>When I ran the above code in my test environment, data was deleted from
some tables that were owned by a different user than what was specified
in the sde connection file. Apparently when tables, feature classes and
datasets are listed it's not just limited to the current user. This
could be particularly problematic if the connection user has permissions
to delete all tables. My solution was to only list tables that started
with the database user followed by a period.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">arcpy</span>
<span class="n">arcpy</span><span class="o">.</span><span class="n">env</span><span class="o">.</span><span class="n">workspace</span> <span class="o">=</span> <span class="sa">r</span><span class="s2">"c:\oracle_connection.sde"</span> <span class="c1"># file that holds all the connection info</span>
<span class="n">desc</span> <span class="o">=</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">Describe</span><span class="p">(</span><span class="n">arcpy</span><span class="o">.</span><span class="n">env</span><span class="o">.</span><span class="n">workspace</span><span class="p">)</span>
<span class="n">dbo</span> <span class="o">=</span> <span class="n">desc</span><span class="o">.</span><span class="n">connectionProperties</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
<span class="n">tables</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListTables</span><span class="p">()</span> <span class="k">if</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span><span class="p">)]</span>
<span class="n">feature_classes</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListFeatureClasses</span><span class="p">()</span> <span class="k">if</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span><span class="p">)]</span>
<span class="n">datasets</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListDatasets</span><span class="p">()</span> <span class="k">if</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span><span class="p">)]</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">tables</span> <span class="o">+</span> <span class="n">feature_classes</span> <span class="o">+</span> <span class="n">datasets</span><span class="p">:</span>
<span class="n">arcpy</span><span class="o">.</span><span class="n">Delete_management</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
</pre></div>
<div class="line-block">
<div class="line">Did you see that? [x.upper() for x in arcpy.ListTables() if
x.upper().startswith(dbo + ".")]</div>
<div class="line">This is what is known as a list comprehension. It's similar to C#'s
LINQ. If this were a C# list, this would be written as var tables =
arcpy.ListTables().Select(x=>x.ToUpper()).Where(x=>x != dbo)</div>
</div>
<p>So what now? Well as it turns out we only wanted to delete pre-approved
tables and feature classes and we were not concerned with datasets. So I
added the tables and feature classes into configuration and I did
intersections with the lists.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">arcpy</span>
<span class="k">def</span> <span class="nf">delete</span><span class="p">(</span><span class="n">tables</span><span class="p">,</span> <span class="n">feature_classes</span><span class="p">):</span>
<span class="sd">"""@param tables: list of unqualified tables that we will drop"""</span>
<span class="sd">"""@param feature_classes: list of unqualified feature classes that we will drop"""</span>
<span class="sd">"""@type tables: [str]"""</span>
<span class="sd">"""@type feature_classes: [str]"""</span>
<span class="n">arcpy</span><span class="o">.</span><span class="n">env</span><span class="o">.</span><span class="n">workspace</span> <span class="o">=</span> <span class="sa">r</span><span class="s2">"c:\oracle_connection.sde"</span> <span class="c1"># file that holds all the connection info</span>
<span class="n">desc</span> <span class="o">=</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">Describe</span><span class="p">(</span><span class="n">arcpy</span><span class="o">.</span><span class="n">env</span><span class="o">.</span><span class="n">workspace</span><span class="p">)</span>
<span class="n">dbo</span> <span class="o">=</span> <span class="n">desc</span><span class="o">.</span><span class="n">connectionProperties</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
<span class="n">deletable_tables</span> <span class="o">=</span> <span class="p">[</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span> <span class="o">+</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">]</span>
<span class="n">deletable_features</span> <span class="o">=</span> <span class="p">[</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span> <span class="o">+</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">feature_classes</span><span class="p">]</span>
<span class="n">tables_in_db</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListTables</span><span class="p">()</span> <span class="k">if</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span><span class="p">)]</span>
<span class="n">features_in_db</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">arcpy</span><span class="o">.</span><span class="n">ListFeatureClasses</span><span class="p">()</span> <span class="k">if</span> <span class="n">x</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">dbo</span> <span class="o">+</span> <span class="s2">"."</span><span class="p">)]</span>
<span class="c1"># if the table is in the list of preapproved tables</span>
<span class="c1"># and the table actually exists</span>
<span class="c1"># we want to delete it</span>
<span class="n">tables_to_delete</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">deletable_tables</span><span class="p">)</span> <span class="o">&</span> <span class="nb">set</span><span class="p">(</span><span class="n">tables_in_db</span><span class="p">))</span>
<span class="n">features_to_delete</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">deletable_features</span><span class="p">)</span> <span class="o">&</span> <span class="nb">set</span><span class="p">(</span><span class="n">features_in_db</span><span class="p">))</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">tables_to_delete</span> <span class="o">+</span> <span class="n">features_to_delete</span><span class="p">:</span>
<span class="n">arcpy</span><span class="o">.</span><span class="n">Delete_management</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
</pre></div>
<p>So there you have it a real world problem solved by list comprehensions
and sets.</p>
</div><!-- /.entry-content -->
<div class="comments">
<h2>Comments !</h2>
<div id="disqus_thread"></div>
<script type="text/javascript">
var disqus_identifier = "python-list-comprehension-and-sets.html";
var disqus_url = "http://bryanprice.net/python-list-comprehension-and-sets.html";
(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = '//bryanpricenet.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
</div>
</article>
</section>
<section id="extras" class="body">
<div class="blogroll">
<h2>blogroll</h2>
<ul>
<li><a href="http://joshuarogers.net">Joshua Rogers</a></li>
</ul>
</div><!-- /.blogroll -->
<div class="social">
<h2>social</h2>
<ul>
<li><a href="http://bryanprice.net/feeds/all.atom.xml" type="application/atom+xml" rel="alternate">atom feed</a></li>
<li><a href="http://twitter.com/southrnprogrmmr">Twitter</a></li>
<li><a href="https://github.com/southernprogrammer">Github</a></li>
</ul>
</div><!-- /.social -->
</section><!-- /#extras -->
<footer id="contentinfo" class="body">
<p>Powered by <a href="http://getpelican.com/">Pelican</a>. Theme <a href="https://github.com/blueicefield/pelican-blueidea/">blueidea</a>, inspired by the default theme.</p>
</footer><!-- /#contentinfo -->
<script type="text/javascript">
var disqus_shortname = 'bryanpricenet';
(function () {
var s = document.createElement('script'); s.async = true;
s.type = 'text/javascript';
s.src = '//' + disqus_shortname + '.disqus.com/count.js';
(document.getElementsByTagName('HEAD')[0] || document.getElementsByTagName('BODY')[0]).appendChild(s);
}());
</script>
</body>
</html>