Commit 8238998a authored by Cristian Weiland's avatar Cristian Weiland

Merge branch 'issue/76' into 'develop'

Issue #76: changing the name of some directories to more intuitive ones

See merge request !38
parents bbcd203e 046c423a
......@@ -13,4 +13,4 @@ O projeto contém dados referentes a:
   ├── workers - Contém scripts para gerenciar dados de Servidores.
   ├── travel_allowance - Contém scripts para gerenciar dados de Diárias de Viagens.
   ├── expenses - Contém scripts para gerenciar dados de Pagamentos.
   └── curl - Contém scripts que auxiliam na utilização do curl.
   └── misc - Contém scripts que auxiliam na utilização do curl.
# Input: The name of a file containing a curl query.
# Output: The output for the curl query.
if [ "$#" -ne 1 ]; then
echo "Usage: $0 <query-file>"
exit
fi
query=$(cat $1)
echo curl -u cw14:123mudar -XGET node1.c3sl.ufpr.br:9200/ufpr-servidores-*/_search?pretty -d \'"$query"\'
......@@ -5,8 +5,8 @@
# Input: Year, month and day from a CSV file, username and password.
# Ex (inserting data from file 20130930_Cadastro.csv): ./create_config.py 2013 09 30 myuser mypassword
# Output: This script will create two config files:
# - JSON: This config will be used for script merge_files_es.py, and will be stored in transparencia/configs/workers/JSON, with its name being config-year-month.
# - Logstash: This config will be used by logstash to insert the resulting CSV from merge_files_es.py into ElasticSearch.
# - JSON: This config will be used for script merge_files.py, and will be stored in transparencia/configs/workers/JSON, with its name being config-year-month.
# - Logstash: This config will be used by logstash to insert the resulting CSV from merge_files.py into ElasticSearch.
import sys, csv, json, math, subprocess
from pathlib import Path
......
......@@ -5,7 +5,7 @@
# This scripts does 4 things:
# 1- Download required files and store them in the right place.
# 2- Create config files via create_config.py
# 3- Merge CSV data and create a new CSV file via merge_files_es.py.
# 3- Merge CSV data and create a new CSV file via merge_files.py.
# 4- Insert CSV file generated in step 2 into ElasticSearch via Logstash.
# Input: Year, Month from CSV file, ElasticSearch's user and password.
......@@ -109,7 +109,7 @@ do
# Step 3:
# Start processing
strReplacement=$( echo "${filter[$key]}" | sed 's/ /\\ /g' )
./merge_files_es.py $path/config-${year}-${month}.json "$strReplacement" "${columnName}"
./merge_files.py $path/config-${year}-${month}.json "$strReplacement" "${columnName}"
rm $path/${year}${month}${day}_Cadastro_Unique.csv
# Step 4:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment