Proxy List

Problem

For a python developer, the list [1, 2, 3] has 3 members, which is true in the python world, but in the “source code modification” world, this list has 5 elements because you have to count the 2 commas. Indeed each comma needs to be taken into account separately because they can have a different formatting.

This makes things quite annoying to deal with because you have to think about the formatting too! For example, if you want to append an item to a list, you need to take care of a lot of details:

  • if the list is empty you don’t have to put a comma
  • otherwise yes
  • but wait, what happens if there is a trailing comma?
  • also, what to do if the list is declared in an indented way (with "\n    " after every comma for example)?
  • etc…

And that’s only for a comma separated list of things: you also have the same formatting details to care about for dot separated lists (e.g. a.b.c().d[plop]) and endl separated lists (a python code block, or you whole source file).

You don’t want to have to deal with this.

Solution

To avoid you to deal with all this boring low level details, RedBaron implements “proxy lists”. This abstraction gives you the impression that the list of things you are dealing with behave the same way than in the python world while taking care of all the low level formatting details.

The “proxy lists” has the same API than a python list so they should be really intuitive to use.

For example:

In [1]: red = RedBaron("[1, 2, 3]")

In [2]: red[0].value.append("42")

In [3]: red
Out[3]: 0   [1, 2, 3, 42]

In [4]: del red[0].value[2]

In [5]: red
Out[5]: 0   [1, 2, 42]

There are, for now, 4 kind of proxy lists:

  • CommaProxyList which handles comma separated lists
  • DotProxyList which handles atomtrailers (those kind of constructions: a.b[plop].c())
  • LineProxyList which handles lines of code (like the body of a function or the whole source code)
  • DecoratorLineProxyList which handles lists of decorators (they are nearly the same as LineProxyList)

Be aware that the proxy list are set on the attribute that is a list, not on the node holding the list. See the ‘value’ attribute access in the examples below.

Usage

As said, proxy lists have the exact same API than python lists (at the exception that they don’t implement the sort and reverse methods). Every method accepts as input the same inputs that you can use to modify a node in RedBaron. This means that you can pass a string containing source code, an FST or a RedBaron node.

Here is a session demonstrating every method of a proxy list:

In [6]: red = RedBaron("[1, 2, 3]")

Please refer to python list documentation if you want to know the exact behavior or those methods (or send a patch to improve this documentation).

append

In [7]: red
Out[7]: 0   [1, 2, 3]

In [8]: red[0].value.append("plop")

In [9]: red
Out[9]: 0   [1, 2, 3, plop]

In [10]: red[0].value
Out[10]: 
0   1
1   2
2   3
3   plop

insert

In [11]: red
Out[11]: 0   [1, 2, 3, plop]

In [12]: red[0].value.insert(1, "42")

In [13]: red
Out[13]: 0   [1, 42, 2, 3, plop]

In [14]: red[0].value
Out[14]: 
0   1
1   42
2   2
3   3
4   plop

extend

In [15]: red
Out[15]: 0   [1, 42, 2, 3, plop]

In [16]: red[0].value.extend(["pif", "paf", "pouf"])

In [17]: red
Out[17]: 0   [1, 42, 2, 3, plop, pif, paf, pouf]

In [18]: red[0].value
Out[18]: 
0   1
1   42
2   2
3   3
4   plop
5   pif
6   paf
7   pouf

pop

In [19]: red
Out[19]: 0   [1, 42, 2, 3, plop, pif, paf, pouf]

In [20]: red[0].value.pop()

In [21]: red
Out[21]: 0   [1, 42, 2, 3, plop, pif, paf]

In [22]: red[0].value
Out[22]: 
0   1
1   42
2   2
3   3
4   plop
5   pif
6   paf

In [23]: red[0].value.pop(3)

In [24]: red
Out[24]: 0   [1, 42, 2, plop, pif, paf]

In [25]: red[0].value
Out[25]: 
0   1
1   42
2   2
3   plop
4   pif
5   paf

__getitem__

In [26]: red
Out[26]: 0   [1, 42, 2, plop, pif, paf]

In [27]: red[0].value
Out[27]: 
0   1
1   42
2   2
3   plop
4   pif
5   paf

In [28]: red[0].value[2]
Out[28]: 2

__setitem__

In [29]: red
Out[29]: 0   [1, 42, 2, plop, pif, paf]

In [30]: red[0].value[2] = "1 + 1"

In [31]: red
Out[31]: 0   [1, 42, 1 + 1, plop, pif, paf]

In [32]: red[0].value
Out[32]: 
0   1
1   42
2   1 + 1
3   plop
4   pif
5   paf

remove

In [33]: red
Out[33]: 0   [1, 42, 1 + 1, plop, pif, paf]

In [34]: red[0].value.remove(red[0].value[2])

In [35]: red
Out[35]: 0   [1, 42, plop, pif, paf]

In [36]: red[0].value
Out[36]: 
0   1
1   42
2   plop
3   pif
4   paf

index

In [37]: red
Out[37]: 0   [1, 42, plop, pif, paf]

In [38]: red[0].value
Out[38]: 
0   1
1   42
2   plop
3   pif
4   paf

In [39]: red[0].value.index(red[0].value[2])
Out[39]: 2

count

In [40]: red
Out[40]: 0   [1, 42, plop, pif, paf]

In [41]: red[0].value
Out[41]: 
0   1
1   42
2   plop
3   pif
4   paf

In [42]: red[0].value.count(red[0].value[2])
Out[42]: 1

len

In [43]: red
Out[43]: 0   [1, 42, plop, pif, paf]

In [44]: red[0].value
Out[44]: 
0   1
1   42
2   plop
3   pif
4   paf

In [45]: len(red[0].value)
Out[45]: 5

__delitem__

In [46]: red
Out[46]: 0   [1, 42, plop, pif, paf]

In [47]: del red[0].value[2]

In [48]: red
Out[48]: 0   [1, 42, pif, paf]

In [49]: red[0].value
Out[49]: 
0   1
1   42
2   pif
3   paf

in

In [50]: red
Out[50]: 0   [1, 42, pif, paf]

In [51]: red[0].value[2] in red[0].value
Out[51]: False

__iter__

In [52]: red
Out[52]: 0   [1, 42, pif, paf]

In [53]: for i in red[0].value:
   ....:     print(i.dumps())
   ....: 
1
42
pif
paf

__getslice__

In [54]: red
Out[54]: 0   [1, 42, pif, paf]

In [55]: red[0].value
Out[55]: 
0   1
1   42
2   pif
3   paf

In [56]: red[0].value[2:4]
Out[56]: 
0   pif
1   paf

__setslice__

In [57]: red
Out[57]: 0   [1, 42, pif, paf]

In [58]: red[0].value[2:4] = ["1 + 1", "a", "b", "c"]

In [59]: red
Out[59]: 0   [1, 42, 1 + 1, a, b, c]

In [60]: red[0].value
Out[60]: 
0   1
1   42
2   1 + 1
3   a
4   b
5   c

__delslice__

In [61]: red
Out[61]: 0   [1, 42, 1 + 1, a, b, c]

In [62]: red[0].value[2:5]
Out[62]: 
0   1 + 1
1   a
2   b

In [63]: del red[0].value[2:5]

In [64]: red
Out[64]: 0   [1, 42, c]

In [65]: red[0].value
Out[65]: 
0   1
1   42
2   c

Access the unproxified node list

The unproxified node list is stored under the attribute node_list of the proxy list. Be aware that, for now, the proxy won’t detect if you directly modify the unproxified node list, this will cause bugs if you modify the unproxified list then use the proxy list directly. So, for now, only use one or the other.

In [66]: red = RedBaron("[1, 2, 3]")

In [67]: red[0].value.node_list
Out[67]: 
0   1
1   , 
2   2
3   , 
4   3

In [68]: red[0].value
Out[68]: 
0   1
1   2
2   3

Omitting “.value”

For convenience, and because this is a super common typo error, if a node has a proxy list on its .value attribute, you can omit to access it and the method access will be automatically redirect to it.

This means that the 2 next lines are equivalent:

In [69]: red[0]
Out[69]: [1, 2, 3]

In [70]: red[0].value.append("plop")

In [71]: red[0].append("plop")

CommaProxyList

CommaProxyList is the most generic and most obvious proxy list, all the examples above are made using it.

It is used everywhere where values are separated by commas.

DotProxyList

DotProxyList is nearly as generic as the CommaProxyList. The specific case of a DotProxyList is that it is intelligent enough to not add a “.” before a “call” ((a, b=c, *d, **e)) or a “getitem” ([foobar]).

In [72]: red = RedBaron("a.b(c).d[e]")

In [73]: red[0].value
Out[73]: 
0   a
1   b
2   (c)
3   d
4   [e]

In [74]: red[0].extend(["[stuff]", "f", "(g, h)"])

In [75]: red[0]
Out[75]: a.b(c).d[e][stuff].f(g, h)

In [76]: red[0].value
Out[76]: 
0   a
1   b
2   (c)
3   d
4   [e]
5   [stuff]
6   f
7   (g, h)

It is used everywhere where values are separated by “.”.

You can see a complete example with a DotProxyList, like for the CommaProxyList, here: DotProxyList usage examples.

LineProxyList

LineProxyList is used to handle lines of code, it takes care to place the correct endl node between and to set the correct indentation and not to break the indentation of the next block (if there is one).

One particularity of LineProxyList is that it shows you explicitly the empty line (while other proxy lists never show you formatting). This is done because you’ll often want to be able to manage those blank lines because you want to put some space in your code or separate group of lines.

In [77]: red = RedBaron("while 42:\n    stuff\n    other_stuff\n\n    there_is_an_empty_line_before_me")

In [78]: red
Out[78]: 
0   while 42:
        stuff
        other_stuff
    
        there_is_an_empty_line_before_me
    

In [79]: red[0].value
Out[79]: 
0   stuff
1   other_stuff
2   '\n    '
3   there_is_an_empty_line_before_me

In [80]: red[0].append("plouf")

In [81]: red
Out[81]: 
0   while 42:
        stuff
        other_stuff
    
        there_is_an_empty_line_before_me
        plouf
    

In [82]: red[0].value
Out[82]: 
0   stuff
1   other_stuff
2   '\n    '
3   there_is_an_empty_line_before_me
4   plouf

You can see a complete example with a LineProxyList, like for the CommaProxyList, here: LineProxyList usage examples.

DecoratorLineProxyList

A DecoratorLineProxyList is exactly the same as a LineProxyList except it has a small modification to indent decorators correctly. Just think of it as a simple LineProxyList and everything will be fine.

Don’t forget to add the :file:`@` when you add a new decorator (omitting it will raise an exception).

Example:

In [83]: red = RedBaron("@plop\ndef stuff():\n    pass\n")

In [84]: red
Out[84]: 
0   @plop
    def stuff():
        pass
    

In [85]: red[0].decorators.append("@plouf")

In [86]: red[0].decorators
Out[86]: 
0   @plop
1   @plouf

In [87]: red
Out[87]: 
0   @plop
    @plouf
    def stuff():
        pass