| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Reading file data from directory |
« View previous topic :: View next topic » |
| Author |
Message
|
| anilmekala |
 Posted: Thu Aug 15, 2019 7:36 am Post subject: Reading file data from directory |
 |
|
Acolyte
Joined: 19 Oct 2012
Posts: 63
|
Hi Team,
I am receiving file from external application with unknown format. I have to read the data record by record from file to pick one of the column element for other process.
I read the file data from FILE INPUT NODE with blob parser and tried to cast character . But it reading as whole file . I need to read record by record to push each record into array.
[1 DATE: JUN 13 19 PGM=L54RSP01 INDIRECT LENDING PROCESSING CENTRE: SDF
TIME: 08:06 BOOKED APPLICATION UPDATE PAGE: 1
------------------------------------------------------------------------------------------------------------------------------------
0 CAPS ACCOUNT LOAN BOOK SRF CO-SRF APPLICANT NAME DEALER
NUMBER NUMBER NO. TRANS NUMBER NUMBER FIRST MIDDLE ID. NAME AMOUNTS
------ ------ --- ----- ------ ------ ----- ------ --- ---- -------
500794144 1550567 1 4993 57766020200 577660210 LAURYSSEN KONANE 12013000 ADAMS CHEVROLET BUI 38,959.12 P
500788799 1550591 1 8493 57766024400 DILLER ODELE 52014500 ABERDEEN MOTORS LTD 35,829.72 P
500793070 1550625 1 5118 57766026900 FRIEDMANN ANTONIE 54099900 RBC NUNAVUT 33,990.11
500794730 1550658 1 8100 57766027700 WESCAM INC 23029700 A. M. FORD SALES LT 45,664.62 P
500789094 1550682 1 4993 WESCAM INC 53000200 M-KINGLAND FORD (HA 26,897.27 P
500794771 1550716 1 8213 57766030100 BRANNAN TEX 75054300 CITY MOTORS 34,802.79
0 --------- -----------
TOTALS 6 216,143.63
1 DATE: JUN 13 19 PGM=L54RSP01 INDIRECT LENDING PROCESSING CENTRE: SDF
TIME: 08:06 BOOKED APPLICATION UPDATE PAGE: 2
------------------------------------------------------------------------------------------------------------------------------------
0 CAPS ACCOUNT LOAN BOOK SRF CO-SRF APPLICANT NAME DEALER
NUMBER NUMBER NO. TRANS NUMBER NUMBER FIRST MIDDLE ID. NAME AMOUNTS
------ ------ --- ----- ------ ------ ----- ------ --- ---- -------
500794144 1550567 1 4993 57766020200 577660210 LAURYSSEN KONANE 38,959.12 P
050079414 50079414 577660228 577660236 SYMES ALEXANDER 38,959.12 L
500788799 1550591 1 8493 57766024400 DILLER ODELE 35,829.72 P
050078879 50078879 577660251 RUDD PADGETT 35,829.72 L
500794730 1550658 1 8100 57766027700 WESCAM INC 45,664.62 P
050079473 50079473 577660285 NUGENT DOROTHEA 45,664.62 L]
I have attached sample file . Please help me if you have any solution .
Regards,
Anil.[/code] |
|
| Back to top |
|
 |
| joebuckeye |
| Posted: Thu Aug 15, 2019 7:57 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
Did you tell the File Input node how the records in the file are delimited?
Or is that still set to 'Whole File'? |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Aug 15, 2019 7:59 am Post subject: Re: Reading file data from directory |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| anilmekala wrote: |
| I have attached sample file . Please help me if you have any solution . |
Assuming the sample is accurate and it's all just text (no packed), build a DFDL model that says the file is an unbounded number records, each record a single string. Set the FileNode to use that model, and read as records delimited by line break.
Push each record into the array you mention.
Simple.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| anilmekala |
| Posted: Thu Aug 15, 2019 8:22 am Post subject: |
|
|
Acolyte
Joined: 19 Oct 2012
Posts: 63
|
Hi Vitor,
Thanks for your reply. As suggested ,I have created DFDL to read each record.
I am getting small error in one of the column data am getting ',' . I want to escape this ,(cama) in body element . Please help me to rectify this error .
Regards,
Anil. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Aug 15, 2019 8:46 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| anilmekala wrote: |
| Please help me to rectify this error . |
Don't have any reference to commas in your model. It sounds like you've used one of the wizards to set up the DFDL; that's not what I said. I said create a model where each record was a single string delimited by a line break because that seemed to be what you wanted to do.
If you've decided to write a DFDL model that reads the records, parsing them to identify the columns you need then good luck to you but that's going to be beyond the scope of the wizard. You'll need to form the model long hand to identify all of the differences and possibilities in data format.
Given that the "file" you're reading is actually not intended to be machine readable but is a human centric printed report (note the '1' and '0' in the first column to throw page and line breaks respectively) it's not a straightforward thing. If you've done this and the only problem you have is a comma, you're doing well.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| timber |
| Posted: Mon Aug 19, 2019 7:55 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
I agree with Vitor - this is quite a tricky format to parse. Easy to parse one example, not easy to get all the edge cases correct.
Please can you supply the following info:
- which column you need to extract (identify which lines in your example)
- how should you reliably identify the lines containing that column
- are all of the lines mandatory? Can some of them get missed out?
- are all of the columns mandatory? Can some of them get missed out?
- what's this problem with the comma? You will probably need to explain with an example.
Without that information, it's very hard to advise on a good solution. |
|
| Back to top |
|
|
| urufberg |
| Posted: Mon Aug 19, 2019 8:07 am Post subject: |
|
|
Apprentice
Joined: 08 Sep 2017
Posts: 28
|
Complementing Vitor's response, it's hard for us to help if we don't know what your final records are and what information must be hold in those records.
As he said, you are working with a file that is not intended to be machine readable, so you are in for a journey.
A suggestion to tackle this problem: try to identify patterns in your file to split your information. Using commas as a record separator is not a good choice in this particular case, but I see you have repeated text, fixed amounts of '-' and numbers and so on...
You need this to build an algorithm that can read your specific file (which again can be painfully hard if the file is not intended to be read by a machine).
Also, focus on the information you really need from the file, there is no point in defining how to read certain records of information you're not gonna use.
Finally, I cannot stress this enough and it might depend on your actual needs, but this seems to be a really hard requirement to fulfill if you're not used to work with DFDL models.
Regards |
|
| Back to top |
|
|
| urufberg |
| Posted: Mon Aug 19, 2019 8:17 am Post subject: |
|
|
Apprentice
Joined: 08 Sep 2017
Posts: 28
|
| timber wrote: |
I agree with Vitor - this is quite a tricky format to parse. Easy to parse one example, not easy to get all the edge cases correct.
Please can you supply the following info:
- which column you need to extract (identify which lines in your example)
- how should you reliably identify the lines containing that column
- are all of the lines mandatory? Can some of them get missed out?
- are all of the columns mandatory? Can some of them get missed out?
- what's this problem with the comma? You will probably need to explain with an example.
Without that information, it's very hard to advise on a good solution. |
Didn't see the response before I posted  . The information requested above is definitely needed and also as they say, you need to make an analysis on the format and not an example.
You can come up with an elegant solution for a couple of examples and then nothing works because you didn't consider 'borders' that were not that border.
Regards |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
