Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stetl bgt improvements #69

Merged
merged 8 commits into from
Feb 27, 2018
Merged

Stetl bgt improvements #69

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
030521c
Make deletion of extracted files optional in ZipFileExtractor
fsteggink Feb 8, 2018
c6f5e5a
Added option to use safe_substitute instead of substitute in StringTe…
fsteggink Feb 8, 2018
2c06228
Added new ExecFilter, which executes a command and returns it output …
fsteggink Feb 8, 2018
bf020b4
Added new RegexFilter, which parses data from a string using a regex …
fsteggink Feb 8, 2018
a5bfef6
Solved some flake8 issues
fsteggink Feb 27, 2018
4d75bba
Small improvements, per Just's recommendations
fsteggink Feb 27, 2018
3db9655
Added unit test for RegexFilter
fsteggink Feb 27, 2018
07670a6
Added unit test for CommandExecFilter
fsteggink Feb 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions stetl/filters/templatingfilter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,13 @@ class StringTemplatingFilter(TemplatingFilter):
consumes=FORMAT.record or FORMAT.record_array, produces=FORMAT.string
"""

@Config(ptype=bool, default=False, required=False)
def safe_substitution(self):
"""
Apply safe substitution?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly add more comment (I did not know e.g. about this standard option in Python Templates), like
if placeholders are missing from mapping and keywords, instead of raising an exception, the original placeholder will appear in the resulting string intact.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. Usually I don't add comments for things which can be easily looked up.

"""
pass

def __init__(self, configdict, section):
TemplatingFilter.__init__(self, configdict, section, consumes=[FORMAT.record, FORMAT.record_array])

Expand All @@ -111,10 +118,16 @@ def create_template(self):
self.template = Template(self.template_string)

def render_template(self, packet):
if type(packet.data) is list:
packet.data = [self.template.substitute(item) for item in packet.data]
if self.safe_substitution:
if type(packet.data) is list:
packet.data = [self.template.safe_substitute(item) for item in packet.data]
else:
packet.data = self.template.safe_substitute(packet.data)
else:
packet.data = self.template.substitute(packet.data)
if type(packet.data) is list:
packet.data = [self.template.substitute(item) for item in packet.data]
else:
packet.data = self.template.substitute(packet.data)

return packet

Expand Down