# Iterate basic question + JSON iteration

**URL:** <https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839>\
**Category:** General Q&A\
**Created:** [September 25, 2023, 9:50am UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839 "2023-09-25T09:50:28Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![David\_Juras](https://community.easymorph.com/user_avatar/community.easymorph.com/david_juras/32/4600_2.png) [@David\_Juras](https://community.easymorph.com/u/David_Juras)\
**Post date:** [September 25, 2023, 9:50am UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/1 "2023-09-25T09:50:29Z")

</div>

Hello guys,

My initial goal: parse a large file embedding Json as a column.  
Initial problem faced: The Json quality is not perfect in the source so that, when I use Json Parse action I can get some errors that do stop the morph. I could not find any option in Json parse to continue processing and just get an error on faulty rows.  
Imagined solution: iterate my table and delegate json parsing to a module so that I can collect error and continue processing (iterate and append results, when an iteration fails -\> Collect errors and continue).  
Enclosed: sample project with my different tests.

Problem 1 - basic iteration: I cannot manage Iteration action to work as expected. Did check examples and community, but my output result always seems empty when using Iterate action (see very simple test trying to inc a column value by 1). I must miss something very simple...

Problem 2 - iterating json (anicipating next step in my process): I always get an error message (colum not found - see dedicated table in the same project). I guess this is due to the fact I am not sending json column as a parameter, but as far as I can see Json parse does not accept parameters as input.

Thanks a lot for your ideas on why I cannot get basic iteration to work (I am feeling a bit noobish here and need it for my EM understanding :-)) and on how I could handle the json processing without interruption.  
[JSON\_PROCESSING\_ITERATION.morph](https://community.easymorph.com/uploads/short-url/9K1qyG1OwoaefJkGUdHYWrxIyn7.morph) (11.1 KB)  
[JSON\_SAMPLE\_PM.csv](https://community.easymorph.com/uploads/short-url/625XMbTXaY3HaUMmio8q1AoBnae.csv) (1.2 KB)  
[TEST\_ITERATION\_JSON.xlsm](https://community.easymorph.com/uploads/short-url/cRYnXhavQY29OW7LUe27PtlvwVM.xlsm) (4.8 KB)

---

<div class="post-metadata">

**Author:** ![David\_Juras](https://community.easymorph.com/user_avatar/community.easymorph.com/david_juras/32/4600_2.png) [@David\_Juras](https://community.easymorph.com/u/David_Juras)\
**Post date:** [September 25, 2023, 1:17pm UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/2 "2023-09-25T13:17:13Z")

</div>

Hello.  
I did find the solution for my first problem (by using the Generate sequence action in called module).  
About problem 2, still investigating and clueless 🙂

---

<div class="post-metadata">

**Author:** ![David\_Juras](https://community.easymorph.com/user_avatar/community.easymorph.com/david_juras/32/4600_2.png) [@David\_Juras](https://community.easymorph.com/u/David_Juras)\
**Post date:** [September 25, 2023, 1:30pm UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/3 "2023-09-25T13:30:58Z")

</div>

Hello again,

For problem 2, I found a (maybe) dirty way. In called module, no Input action, I pass as parameter the json column, Generate a sequence with 1 line, calculate a new column using an expression that takes the value of module parameter, process, send back result.  
Is there any better or more straighforward way to achieve the goal ?

Thanks and regards,

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [September 26, 2023, 1:57am UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/4 "2023-09-26T01:57:18Z")

</div>

Hi David,

I've edited your project to make it work (see below):

[JSON\_PROCESSING\_ITERATION.morph](https://community.easymorph.com/uploads/short-url/7QXm2M0HokaJ0EFaLHF02kvFgSj.morph) (12.3 KB)

Main changes:

- Removed (disabled) the "Input" action. This action is not required for the regular "Iterate" action because it passes row values as parameters, not as an entire dataset.
- Added parameters in modules ITERATE and ITERATE\_JSON, populated them with sample values
- Added the "Parameter table" action. It's a way to make a table from module parameters. I could do the same with expressions, just in a less convenient way.

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [September 26, 2023, 2:08am UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/5 "2023-09-26T02:08:25Z")

</div>

Also, it doesn't look like iterations are required here. You can use the [new "Extract JSON properties" action](https://help.easymorph.com/doku.php?id=transformations:extractjson). It has a failover setting. See the updated project below, I've added the "No iteration" table.

[JSON\_PROCESSING\_ITERATION.morph](https://community.easymorph.com/uploads/short-url/fLQBNiZs3dPxkrs6o6Kg5awpFnf.morph) (14.0 KB)

 ![image](https://community.easymorph.com/uploads/default/original/2X/8/8b6dfb59fbb2d002dede94eeca209f36d5c429ed.png)

---

<div class="post-metadata">

**Author:** ![David\_Juras](https://community.easymorph.com/user_avatar/community.easymorph.com/david_juras/32/4600_2.png) [@David\_Juras](https://community.easymorph.com/u/David_Juras)\
**Post date:** [September 26, 2023, 2:53pm UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/7 "2023-09-26T14:53:03Z")

</div>

![ExtractJSONProperties_NoInDepthProperties](https://community.easymorph.com/uploads/default/original/2X/6/6a5bc65a1a096ec2e38fa76ef327f6a378213329.png)  
[JSON\_PROCESSING\_ITERATION.morph](https://community.easymorph.com/uploads/short-url/29YJAW63lLWJhycGdKcZIPX8EJ9.morph) (15.1 KB)  
[Json\_WithFaultyLine.dset](https://community.easymorph.com/uploads/short-url/geJclV5Fwb1qIweHNUgY1V87oRx.dset) (333.2 KB)  
 ![ParseJson_AllPropertiesDisplayed](https://community.easymorph.com/uploads/default/original/2X/4/480896c787d62b081473ba24376eb46848f6969d.png)

---

<div class="post-metadata">

**Author:** ![David\_Juras](https://community.easymorph.com/user_avatar/community.easymorph.com/david_juras/32/4600_2.png) [@David\_Juras](https://community.easymorph.com/u/David_Juras)\
**Post date:** [September 26, 2023, 2:54pm UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/8 "2023-09-26T14:54:05Z")

</div>

Hello Dmitry, (looks like my previous post was cut when including attachments, sorry)

Thanks for the tip with parameter table.

About Extract json properties new action, it fails when I am using it. I did attach new version of the project with No iteration Test with faulty json fails table. And provided a dset with some working rows and last faulty row on a different json I need to parse as well (csv export won't work as my json is sometimes far bigger than the 32K or so caracters limitation that seem to apply to csv export action).

As well, I need to parse deeper on this additional Jsons, I can do it with the Parse Json action (see screenshot enclosed) but couldn't find a way with the Extract JSON properties new action. So maybe I will have to iterate eventually...

Thanks a lot for your support.

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [September 26, 2023, 7:11pm UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/9 "2023-09-26T19:11:56Z")

</div>

In the column "JSON response", the text value in the last line (RowNo 1229) is not a valid JSON object. Its JSON structure is broken. That's why the action fails. Maybe try contacting the producer of the JSON data and ask them to fix it.

You can test whether a text value is a valid JSON object or not using the [isjson()](https://help.easymorph.com/doku.php?id=syntax:functions:isjson) function. After that, you can filter rows with valid JSONs and parse them in one table, and rows with invalid JSONs in another table (and decide what to do with them).

 ![image](https://community.easymorph.com/uploads/default/original/2X/b/be1dcff2657392b04005b31e7d4c662f52b12ace.png)

Here is the updated project (notice the new tab "Group 2").  
[JSON\_PROCESSING\_ITERATION.morph](https://community.easymorph.com/uploads/short-url/9KbqMYYMNBbe3MjeUJTPqx5vm5g.morph) (16.8 KB)

---

<div class="post-metadata">

**Author:** ![David\_Juras](https://community.easymorph.com/user_avatar/community.easymorph.com/david_juras/32/4600_2.png) [@David\_Juras](https://community.easymorph.com/u/David_Juras)\
**Post date:** [September 27, 2023, 4:14am UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/10 "2023-09-27T04:14:34Z")

</div>

Thanks a lot Dmitry for all those precious inputs and tips. I will surely use them in many projects now.  
About my specific use case, and as I need to operate many additional actions after json operations on large files (many Gb) with huge JSONs (sometimes over 1M caracters....), the iteration approach is finally the most performant.  
I did some testing on a 600 Mb sample and iteration approach, as it splits operations to very small chunck, is at least 3 times faster than a oneshot.  
Regards

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [September 27, 2023, 12:58pm UTC](https://community.easymorph.com/t/iterate-basic-question-json-iteration/4839/11 "2023-09-27T12:58:45Z")

</div>

You're welcome 🙂
