Disable lexing for specific fields

[sjopet]

I'm trying to upgrade from v2 to v3 due to an issue with being unable to escape a string value that was interpreted as a variable. However I now find myself in the exact opposite situation of having to escape all the html tags, percent signs, brackets etc. For some of my classes this is simply not viable as it would mean i would have to add thousands of backwards \ slashes to my fixtures and falling back to v2 would be my only real option.

The only solution I see is to disable the parsing/lexing of certain fields of particular classes which I know will never hold variables functions or references Alice should worry about. The problem is that I have no idea how to achieve this or if it's even remotely possible.

Any ideas or help would be greatly appreciated

[theofidry]

I tried that but couldn't find any easy way for now. We could either have a new markup like:

foo: <<<'ESCAPED' ... ESCAPED

or something like that and then in the lexer/parser bypass that expression. An alternative which is already available is to leverage the faker providers to return the string you want which won't be evaluated

[sjopet]

That's the only existing 'solution' I can find so far, It looks to me as reversing the problem though; having to add Alice/Faker directives to all the things I don't want Alice or Faker to touch.

In both cases I would still have to alter all the fields the lexer raises exceptions for, which would still mean more then a thousand cases.

[theofidry]

In both cases I would still have to alter all the fields

Unfortunately yes. It was pure luck that it was working fine in 2.x :/

[pscheit]

How can I escape everything in general?

I have a string like:

containerDefinitionsJson: "[\n    {\n        \"position\": [\n            0,\n            0\n        ],\n        \"dimension\": [\n            100,\n            100\n        ],\n        \"style\": \"\",\n        \"type\": \"image\",\n        \"slot\": 1\n    },\n    {\n        \"position\": [\n            5,\n            5\n        ],\n        \"dimension\": [\n            40,\n            30\n        ],\n        \"style\": \"\",\n        \"type\": \"headline\",\n        \"slot\": 2\n    }\n]"

I tried "\\[" and "\[" to escape it, but it does not work.
Is there currently any workaround?

Nelmio\Alice\Throwable\Exception\FixtureBuilder\ExpressionLanguage\LexException: Could not lex the value "\[
    {
        "position": [
            0,
            0
        ],
        "dimension": [
            100,
            100
        ],
        "style": "",
        "type": "image",
        "slot": 1
    },
    {
        "position": [
            5,
            5
        ],
        "dimension": [
            40,
            30
        ],
        "style": "",
        "type": "headline",
        "slot": 2
    }
]".

/app/vendor/nelmio/alice/src/Throwable/Exception/FixtureBuilder/ExpressionLanguage/ExpressionLanguageExceptionFactory.php:69

[theofidry]

You could put your string as a returned value of a faker data provider for now

[floplus]

An other workaround, which worked (for me):

parameters:
  param1: 'App\Entity\E1'

App\Entity\Abc:
   abc1:
    name: "Abc"
    __calls:
      - setClassName: [<{param1}>]

what didn't work was (not even with different levels of escaping the backslash):

App\Entity\Abc:
   abc1:
    name: "Abc"
    className: 'App\Entity\E1'

[theofidry]

@floplus I would open a dedicated issue for that one. It looks like a bug that can be dealt with.

Also I think regardless of the progress of that issue meanwhile we could add a doc entry for it

[tifabien]

We also need to disable the lexer when a field contains the $ symbol

[theofidry]

@tifabien what do you mean?

[tifabien]

@theofidry oops I was to vague sorry. I meant that we got also an error when in your fixture you set for example something like

App\Entity\Currency:
    dollar:
        symbol: $

[theofidry]

$ has a special meaning for alice so you need to escape it: \$

[tifabien]

Thanks @theofidry, I'll escape it but is it something new with alice 3.x because I didn't have to escape it with the previous 2.x version?

[theofidry]

Then it's a user-land BC break and this could be fixed. I think it's reasonable to expect $ alone or with trailing spaces to not be interpreted as a variable

[loic425]

To use an array in a json string, you can use this workaround.

parameters:
     data: '{"name":"Netflix", "types":["streeming", "video"]}'

App\Entity\Test:
    test_1:
        __calls:
          - setData: [<{data}>]

[yakobe]

I'm currently also having issues with strings being lexed when they shouldn't. Adding them all as parameters could get very verbose.

What is the currently recommended way to solve it? Is there any documentation about it anywhere? How can people help?

[theofidry]

@yakobe most strings should be escapable with \ as documented. Sometimes the lexer may be wrong in which case you can either do the parameters workaround, have a custom faker provider or attempt to fix it in alice (see LexerIntegrationTest and ParserIntegrationTest).

That said there is a release coming where alice just let the string be a string if it's an invalid expression (instead of currently bailing out)

[yakobe]

That said there is a release coming where alice just let the string be a string if it's an invalid expression (instead of currently bailing out)

@theofidry that would be awesome. Any idea when it is due? Or is there a pull request i can follow or something?

[theofidry]

I need to test out #938 and then it will be ready to tag

[yakobe]

Thank you! Solves my problems perfectly 👍.

[titomiguelcosta]

any updates @theofidry ? it would be nice to get it merged

[theofidry]

I’ll check it out once I’m back from holidays.

…

On Fri 26 Apr 2019 at 08:30, Tito Miguel Costa ***@***.***> wrote: any updates @theofidry <https://github.com/theofidry> ? it would be nice to get it merged — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#840 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABHPVAN3UKX3OMP2BFTWOZLPSI5KFANCNFSM4EIBJLZA> .

[tacman]

Is that out? I'm still getting the problem with something like

     summary: "After putting $18 million into a campaign"

It works with

     summary: <identity("After putting $18 million into a campaign to try to trick San Francisco voters in")>

But this means escaping (with <identity()>) every string in my generated files. So simply loading the string when it fails would be preferable.

[theofidry]

I first attempted that but it turned out it was too flimsy: a lot of people ended up with strings where they did not expect it because the error was silenced.

There is currently no proper escaping. The identity can work to some extend, but there is a few cases e.g. with nested calls that might not work.

In any case, I think it's something that would be better solved with #998

[c33s]

this feature would be really important, because escaping all links in a multiline markdown text can't be the right solution.

App\Entity\Item:
    item_1
        __construct:
            visible: true
            rank: 50
            text: >
                example text
                more text [markdown link with reference][1]
                
                [1]: https://github.com/nelmio/alice/issues/new

an idea would be to have an exclude list for such things. fiddeling around with the yaml format feels counterproductive for me. having a separate file like:

App\Entity\Item:
    - text

or setting it in php in the fixture loader could be a solution.

[theofidry]

Maybe you can achieve it via the parameters which are used as is and not parsed

[tacman]

Yes, it's _possible_, but using parameters to achieve this is terribly messy and convoluted. Or maybe I'm missing something, can you provide an example of how to do that, one that The inability to escape field values is the biggest reason I can't use this bundle, so would be thrilled if that were added. Thanks.

…

On Mon, Aug 10, 2020 at 3:44 PM Théo FIDRY ***@***.***> wrote: Maybe you can achieve it via the parameters which are used as is and not parsed — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#840 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEXIQMHAJJH7UMB746PJ5DSABEYRANCNFSM4EIBJLZA> .

[theofidry]

Yes, it's possible, but using parameters to achieve this is terribly
messy and convoluted. Or maybe I'm missing something, can you provide an
example of how to do that, one that

When loading the fixtures, you can always pass objects & parameters, their keys making them accessible from the fixtures. I agree it's not the best experience for sure, and if you use it extensively maybe a loader decorator could help to make your usage simpler.

I explored a few ways, but I couldn't come with a really robust an nice way, hence I why I think #998 is really the way forward

[c33s]

#998 is not really helpful, if i would like to use php for fixture definition i simply would stick to doctrine fixtures (i use alice for "real" fixtures and also for development fixtures).

the main benefit using alice for me, next to have a faker integration, is that i can let non programmers write some fixtures.

it is unhandy enough that i have to use __construct and __calls where the order of the parameters matter. it is even unhandy for a developer. errors like must be of the type string, object given happen easily. but factory or construct calls are required if you want to use constuctor based entities without tons of setters. calls are required if you have a nondeveloper who writes fixtures. for a normal human, images belonging to an item are defined in the item, not in a different file. the assoc between customer and item is easier to understand and it feels ok to have them in a different file but the images are handyier inside the items file.

in the end it looks like that:

App\Item:
    item_template (template):
        visible: false
        rank: 50
    item_1 (extends _item_template):
        __construct:
            name: myname
            from_date: '<date_create_immutable(2018-01-01)>'
            client: '@client_7'
            visible: true
            text: >
                lorem ipsum
        __calls:
            - addImagesFromArray:
                -
                    -
                        name: headline
                        file: screenshot_website.jpg
                    -
                        name: headline
                        sub_headline: subheadline
                        file: anotherfile.jpg

is quite ugly to read for a non developer, the double array of the __calls the impossibility of using markdown in text, the specific order requirement every property using non parsed parameters doesn't make it better.

much cooler would be something like the following (just wip thought not a real final solution. should only line out what i mean)

config.yaml or config.php (does not matter as the config has to come from a develeoper)

App\Item:
    map_fields_to_constructor: true
    unparsed_fields:
        - text
        - name
        - visible
    call_mapping:
        images:
            method: addImagesFromArray
            xyz: #some kind of parameter mapping,  where i can define the array/data defined in `images` should be used as one parameter, or each image should be its own parameter,...

items_fixture.yaml

App\Item:
    item_template (template):
        visible: false
        rank: 50
    item_1 (extends _item_template):
        name: myname
        from_date: '<date_create_immutable(2018-01-01)>'
        client: '@client_7'
        visible: true
        text: >
            lorem ipsum
        images:
            -
                name: headline
                file: screenshot_website.jpg
            -
                name: headline
                sub_headline: subheadline
                file: anotherfile.jpg

[Eimantas123]

Still not possible to disable string parsing/lexing?

[tacman]

When loading the fixtures, you can always pass objects & parameters, their keys making them accessible from the fixtures.

Can you point me to the documentation on how to do that? I'm still not following, and not being able to store values with special symbols ($, %, etc.) is a show-stopper.