GUI editor to tame mod_security rules

⌈⌋ ⎇ branch:  modseccfg


Check-in [950ea0eb0c]

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Performance fix for pyrewrite in range() check.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 950ea0eb0c23653ecfc52761143fef4c68adca4b2b9c72cb6e70a419bee89c0f
User & Date: mario 2021-03-29 21:23:05
Context
2021-03-29
21:24
Shorten titles (rm modseccfg:) check-in: c7b7b39b4f user: mario tags: trunk
21:23
Performance fix for pyrewrite in range() check. check-in: 950ea0eb0c user: mario tags: trunk
2021-03-26
12:13
html2mallard update: support direct .md conversion, and http:// url params, doc updates. check-in: ada19bd287 user: mario tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Added modseccfg/data/help/pyrewrite.md.

































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
# MSC_PyRewrite

There's a plugin for rewriting core rules files (*.conf) per
[msc_pyparser](https://pypi.org/project/msc-pyparser/). It
can be enabled in [Settings], and then appears directly in
the File menu.

This plugin bundles a handful of examples, as listed in the
"Transform" dropdown. But you can also edit the code block
before running.


## Usage

In the dialog, you can limit the operation to specific
secrule *.conf files and optionally id ranges. Then there's
a dropdown for default samples, and the actual code editor.

  * So first select the files you want to have rewritten.
    They must toggled on by one.
  * Optionally set a range to limit the processing to SecRules.
    The format is `0-9999999` to defined a range. Clear the
    field if you want to apply the changes to any and all rules
    instead.
  * Then pick a transform script to apply on each rule or file.
    The selection will update/overwrite the code block input.
  * The shown scripts can be adapted before executing them.
    What you see in the editor box is what will actually run.
    So it must be valid Python code with proper indentation.

The "Run/Rewrite" button will execute that code block. Ideally
you want to leave the `--dry-run` mode enabled first, to see
if it runs into any errors. In this mode the files aren't
rewritten, but potentially changes brought up in a preview
window only.

Note that any print() output will appear on the console/terminal
where modseccfg was started from.


##  Operation modes


### Rule-by-rule

All default snippets only change properties on a rule by
rule basis. They typically utilize the `rule.function()`
convenience methods.

But you can also use the raw `r` dictionary.


### Block-wise

Alternatively, there's the block-wise mode, which might allow
some msc_parser transform functions to be used literally.
Simply prefix the code block with `# type: block` to mark up
such scripts.

Might be possible to utilize the examples/ from msc_pyparser
directly, since the Transform function is often suitable
for direct use as block-wise script.

For example the [example04_reduce_escs.py](https://github.com/digitalwave/msc_pyparser/blob/master/examples/example04_reduce_escs.py)
contains a "Transform" class. Just copy that into the input box.
And append an instantiation and inocation call:

    # type: block

    # copy verbatim
    class Transform(object):
        def __init__(self, data):
	...

    # execute
    Transform(self.data).stripescape()

Where `self.data` is the raw parser structure, and `.stripescape()`
the target method in this example/class.


### Available variables

You can see the "help" Transform sample for some notes:

   * `r` is the plain msc_pyparser dict for a rule (alias: `d`)
       * r["type"] is often "SecRule"
       * r["actions"] a list of dicts (alias: `actions`)
   * `rule` is a wrapper object with some convenience methods
       * rule.id()
       * rule.has_tag("app")
       * rule.add_tag("my_new_tag")
       * for l in rule.find_actions("logdata"):
   * `parser` is the current MSCParser instance
   * `self.data` is parser.configlines

Added modseccfg/data/help/pyrewrite.page.









































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
<page xmlns="http://projectmallard.org/1.0/"
 type="guide" id="pyrewrite">

<info>
    <link type="guide" xref="index#nav"/>

    <desc></desc>
    <?http header="X-Generator: html2mallard" ?>
</info>

<title>MSC_PyRewrite</title>


<section>
 <title>MSC_PyRewrite</title>
 <p>There's a plugin for rewriting core rules files (*.conf) per
 <link type="seealso" href="https://pypi.org/project/msc-pyparser/">msc_pyparser</link>. It
 can be enabled in [Settings], and then appears directly in
 the File menu.</p>
 <p>This plugin bundles a handful of examples, as listed in the
 "Transform" dropdown. But you can also edit the code block
 before running.</p>
</section>

<section>
 <title>Usage</title>
 <p>In the dialog, you can limit the operation to specific
 secrule *.conf files and optionally id ranges. Then there's
 a dropdown for default samples, and the actual code editor.</p>
 <list>
 <item><p>So first select the files you want to have rewritten.
     They must toggled on by one.</p></item>
 <item><p>Optionally set a range to limit the processing to SecRules.
     The format is <code>0-9999999</code> to defined a range. Clear the
     field if you want to apply the changes to any and all rules
     instead.</p></item>
 <item><p>Then pick a transform script to apply on each rule or file.
     The selection will update/overwrite the code block input.</p></item>
 <item><p>The shown scripts can be adapted before executing them.
     What you see in the editor box is what will actually run.
     So it must be valid Python code with proper indentation.</p></item>
 </list>
 <p>The "Run/Rewrite" button will execute that code block. Ideally
 you want to leave the <code>--dry-run</code> mode enabled first, to see
 if it runs into any errors. In this mode the files aren't
 rewritten, but potentially changes brought up in a preview
 window only.</p>
 <p>Note that any print() output will appear on the console/terminal
 where modseccfg was started from.</p>
</section>

<section>
 <title>Operation modes</title>
</section>

<section>
 <subtitle>Rule-by-rule</subtitle>
 <p>All default snippets only change properties on a rule by
 rule basis. They typically utilize the <code>rule.function()</code>
 convenience methods.</p>
 <p>But you can also use the raw <code>r</code> dictionary.</p>
</section>

<section>
 <subtitle>Block-wise</subtitle>
 <p>Alternatively, there's the block-wise mode, which might allow
 some msc_parser transform functions to be used literally.
 Simply prefix the code block with <code># type: block</code> to mark up
 such scripts.</p>
 <p>Might be possible to utilize the examples/ from msc_pyparser
 directly, since the Transform function is often suitable
 for direct use as block-wise script.</p>
 <p>For example the <link type="seealso" href="https://github.com/digitalwave/msc_pyparser/blob/master/examples/example04_reduce_escs.py">example04_reduce_escs.py</link>
 contains a "Transform" class. Just copy that into the input box.
 And append an instantiation and inocation call:</p>
 <code># type: block

 # copy verbatim
 class Transform(object):
     def __init__(self, data):
 ...

 # execute
 Transform(self.data).stripescape()
 </code>
 <p>Where <code>self.data</code> is the raw parser structure, and <code>.stripescape()</code>
 the target method in this example/class.</p>
</section>

<section>
 <subtitle>Available variables</subtitle>
 <p>You can see the "help" Transform sample for some notes:</p>
 <list>
 <item><p><code>r</code> is the plain msc_pyparser dict for a rule (alias: <code>d</code>)</p>
 <list>
  <item><p>r["type"] is often "SecRule"</p></item>
  <item><p>r["actions"] a list of dicts (alias: <code>actions</code>)</p></item>
 </list>
 </item>

 <item><p><code>rule</code> is a wrapper object with some convenience methods</p>
  <list>
  <item><p>rule.id()</p></item>
  <item><p>rule.has_tag("app")</p></item>
  <item><p>rule.add_tag("my_new_tag")</p></item>
  <item><p>for l in rule.find_actions("logdata"):</p></item>
  </list>
 </item>

 <item><p><code>parser</code> is the current MSCParser instance</p></item>
 <item><p><code>self.data</code> is parser.configlines</p></item>
 </list>
</section>


</page>

Changes to modseccfg/msc_pyrewrite.py.

141
142
143
144
145
146
147

148
149
150
151
152
153
154
                # empty line after the rule
                self.lineno += 1
            else:
                d['lineno'] = self.lineno
                self.lineno += 1
    """,
}


class rulewrap():
    """ utility functions to simplify adapting rules and actions """
    
    def __init__(self, r):
        self.r = r








>







141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
                # empty line after the rule
                self.lineno += 1
            else:
                d['lineno'] = self.lineno
                self.lineno += 1
    """,
}


class rulewrap():
    """ utility functions to simplify adapting rules and actions """
    
    def __init__(self, r):
        self.r = r

183
184
185
186
187
188
189







190
191
192
193
194
195
196
                yield a["act_arg"]

    def id(self):
        """ find_action("id"), return as integer, or None """
        for a in self.r["actions"]:
            if a['act_name'] == "id":
                return int(a['act_arg'])









class show():
    """ hook to file menu in mainwindow """

    def __init__(self, mainwindow, data, *a, **kw):
    







>
>
>
>
>
>
>







184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
                yield a["act_arg"]

    def id(self):
        """ find_action("id"), return as integer, or None """
        for a in self.r["actions"]:
            if a['act_name'] == "id":
                return int(a['act_arg'])
    
    def within(self, limit):
        id = self.id()
        if not id:
            return False
        if limit.start <= id <= limit.stop:
            return True


class show():
    """ hook to file menu in mainwindow """

    def __init__(self, mainwindow, data, *a, **kw):
    
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
                p(500)
            else:
                for r in parser.configlines:
                    p(2)
                    rule = rulewrap(r)
                    if r["type"] != "SecRule":
                        continue
                    if limit and rule.id() not in limit:
                        continue
                    # aliases (msc_pyparser examples)
                    d = r
                    actions = r["actions"]
                    # eval script block
                    exec(script)
                    p(10)







|







257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
                p(500)
            else:
                for r in parser.configlines:
                    p(2)
                    rule = rulewrap(r)
                    if r["type"] != "SecRule":
                        continue
                    if limit and not rule.within(limit):
                        continue
                    # aliases (msc_pyparser examples)
                    d = r
                    actions = r["actions"]
                    # eval script block
                    exec(script)
                    p(10)