Pessoal,
a versao final do technote que estava escrevendo sobre processos de migracao utilizando o HPL como ferramenta de carga e descarga (http://the-information-x.blogspot.com.br/2012/04/novo-artigo-technotedevworks-sobre.html), ficou pronta, foi aprovada e publicada, veja no link abaixo:
http://www.ibm.com/support/docview.wss?uid=swg21587169
Setting up High-Performance Loader (HPL) in the Informix command line.
Technote (FAQ)
Question
How do I set up the High-Performance Loader (HPL) Informix utility?
Cause
Informix
DBAs and System Administrators usually find HPL a little bit tricky,
due to its own shared memory management engine and specific command line
instructions to handle the load and unload jobs control.
This document gathers a compilation of useful links and ease-of-use instructions to quickly set HPL ready to run.
Answer
Eventually,
some may find the GUI option the best path, but this document intends
to prepare the reader to set HPL in just a few steps, thru the command
line interface.
1. First of all, you should choose a table to create the HPL job (and the onpload database):
$ onpladm create job j_items -d "items.unl" -D stores -t items
Successfully created Job j_items
where:
- j_items -> job name for the table
- items.unl -> flat file to receive the unloaded table rows
- stores -> database name
- items -> table name
As soon as the first job is created, the onpload database is created too, if is doens't exist already.
If it does exist, you should rename it so it won't cause any trouble.
2. To execute the j_items job as an unload job, just issue the command below, using the -fu flag:
$ onpload -j j_items -fu -l items.log -i 100000 -R $PWD/a
where:
- $PWD -> environment variable created to store the current directory value
- j_items -> job name for the table
- items.log -> error log file name
3. To use the same job as a load job, just issue the command below, changing the -fu flag by the -fl flag:
$ onpload -j j_items -fl -l items.log -i 100000 -R $PWD/a
4. Obviously, no one would use HPL facility to unload/load a
single table. The sections bellow show some SQL queries to generate the
onpload commands for each table. You can use the DBAccess utility to
execute these commands.
Don't forget to change the <dbname> variable for your database name, i.e.: stores_demo.
- This query, generates the CREATE JOB command thru the onpladm facility for all tables in the database, in Express Mode:
select tabname from systables where tabid > 99 and tabtype = "T" into temp a;
output to hpl_create_job.sh without headings
select "onpladm create job job_"||trim(tabname)||" -d
'/tmp/"||trim(tabname)||".unl' -D <dbname> -t "||trim(tabname)
from a
- This query, generates the CREATE JOB command thru the onpladm facility for all tables in the database, in Deluxe Mode:
select tabname from systables where tabid > 99 and tabtype = "T" into temp a;
output to hpl_create_job_delux.sh without headings
select "onpladm create job job_"||trim(tabname)||" -d
'/tmp/"||trim(tabname)||".unl' -D <dbname> -t "||trim(tabname)||"
-fc" from a
- This query, generates the job load command thru the onpload facility for all tables in the database:
select tabname from systables where tabid > 99 and tabtype = "T" into temp a;
output to hpl_load_job.sh without headings
select "onpload -j job_"||trim(tabname)||" -fl ","-l /tmp/"||trim(tabname)||".log -i 100000 -R /tmp/"||trim(tabname) from a
- This query, generates the job unload command thru the onpload facility for all tables in the database:
select tabname from systables where tabid > 99 and tabtype = "T" into temp a;
output to hpl_unload_job.sh without headings
select "onpload -j job_"||trim(tabname)||" -fu ","-l
/tmp/"||trim(tabname)||".log -i 100000 -R /tmp/"||trim(tabname) from
a
For all the tables that has BLOB columns only Deluxe Mode can be used.
5. When all the shell scripts are done, you can clean them up,
avoiding line feeds and weird characters which may cause a failure at
execution time.
Follows attached, two scripts that might be used as cleaning tools:
a. Use this tool to remove line feeds, name it join.sh:
:join
/\\$/{N
s/\\\n/ /
b join
}
b. Use this tool to remove blank lines, name it del_blank.sh:
Here's an example on its use:
cat hpl_load_job.sh | sed -f join.sh > zz
sh del_blank.sh zz > zzz
mv zzz hpl_create_job.sh
And the final hpl_create_job.sh script is clean.
4. One important notice that should be taken care of is the following HPL environment variables:
- PLOAD_SHMAT:
http://publib.boulder.ibm.com/infocenter/idshelp/v115/topic/com.ibm.hpl.doc/hpl011066502.htm
When you set the PLOAD_SHMAT environment variable, the pload converter
calculates the address using a global attached segment list that is
maintained across pload virtual processors. The pload converter attaches
at the next available address after the highest address on the list,
ensuring that the converter always attaches to an unused shared memory
segment.
Error messages like this can be seen in the online.log file when PLOAD_SHMAT is not set:
15:29:46 PLOAD aborting: cpu VP 15 could not attach shared memory at
the required address. Set the environment variable PLOAD_SHMAT to 1 and
try again.
- IFX_XFER_SHMBASE
http://publib.boulder.ibm.com/infocenter/idshelp/v115/topic/com.ibm.sqlr.doc/ids_sqr_255.htm
After the database server allocates shared memory, the database server
might allocate multiple contiguous OS shared memory segments. The client
utility that connects to shared memory must attach all those OS segments
contiguously also. The utility might have some other shared objects (for
example, the xbsa library in onbar) loaded at the address where the
server has shared memory segment attached. To workaround this situation,
you can specify a different base address in the environment variable
IFX_XFER_SHMBASE for the utility to attach the shared memory segments.
The following error message can be seen in the online.log file:
15:29:46 shmat: [22]: operating system error
15:29:46 Client could not attach server shared memory segment, use IFX_XFER_SHMBASE.
Criticas e sugestoes sao aceitas (e rating positivo tambem...rs!!)
Espero que gostem.
Abracos,
Fernando [FFO]
