Commit a6949283 authored by Edu Trevisan's avatar Edu Trevisan Committed by Odair M.

#143 Merge 'production' branch into 'development'

parent b6c0c398
......@@ -14,10 +14,10 @@ base_dados
src/.coverage
# lixo
static/*
# lixo
src/static
**/migrations/**
*~
*.swp
......
# ADEGA
Este software faz parte de um projeto do PET Computação UFPR para
Este software faz parte de um projeto do PET Computação UFPR para
análise de dados dos cursos de graduação da UFPR. Veja a [wiki](http://gitlab.c3sl.ufpr.br/adega/adega/wikis/home).
......@@ -80,6 +80,37 @@ $ sudo make docker-remove-all
```
*Observação*: Esse comando **não** irá deletar qualquer arquivo do projeto / diretório local, apenas os containers.
## Versão de produção
Para fazer o deploy do adega na versão de produção primeiro verifique se settings.py está com a seguinte linha:
```python
DEBUG = False
```
Então execute:
```bash
$ sudo make docker-production
```
Este comando funciona da mesma forma que `make docker-up` e portando também funciona com os comandos `make docker-manage`.
O aplicativo vai rodar na porta 8000, para alterar mude a porta do container nginx no arquivo `docker-production.yml`.
### Observações do servidor
Se não for possível construir as imgens do docker no servidor será necessário copia-las manualmente por scp.
Para salvar uma imagem execute:
```bash
$ sudo docker save imagem -o imagem_destino
```
Para carregar:
```bash
$ sudo docker load -i imagem
```
É necessário carregar as imagens `adega_web`, `adega_db` e `adega_nginx`.
Se alterações forem feitas no código do adega elas serão automaticamente refletidas no servidor, mesmo na versão de produção.
Porém se mudanças forem feitas no container do nginx a imagem deste deverá ser refeita.
Para manter o aplicativo atualizado só é necessário dar pull na branch production.
## Instalação e dependências manuais (não recomendado)
......@@ -96,8 +127,8 @@ sudo -u postgres psql < postgres/create.sql
```
se você possui o arquivo do banco de dados compartilhado internamente pelos
desenvolvedores do projeto coloque-o na home do projeto, ele vem com um usuário
se você possui o arquivo do banco de dados compartilhado internamente pelos
desenvolvedores do projeto coloque-o na home do projeto, ele vem com um usuário
`pet` com senha `pet` pré-configurado para testes.
......@@ -130,7 +161,7 @@ Ao sair do projeto execute `exit` para sair do virtualenv e evitar polui-lo
## Transformando o seu usuário em um professor
Após você logar no sistema com o seu super usuário você terá acesso ao `URL_DO_SITE/admin`, graças ao [Django admin](https://docs.djangoproject.com/en/1.10/ref/contrib/admin/) nesta tela você é capaz de gerenciar os dados salvos nas models do projeto.
Para transformar o seu usuário em professor basta clicar em `professor`e então selecionar o seu usuário e o curso. Agora se você voltar para a página inicial do sistema você deve ver uma listagem dos seus cursos.
Para transformar o seu usuário em professor basta clicar em `professor`e então selecionar o seu usuário e o curso. Agora se você voltar para a página inicial do sistema você deve ver uma listagem dos seus cursos.
## Executando análises (se vocẽ está usando docker)
......
......@@ -9,7 +9,7 @@ services:
- POSTGRES_DB=adega
web:
container_name: adega_web_1
build:
build:
context: .
dockerfile: ./docker_scripts/Dockerfile
command: bash "./docker_scripts/on_docker_init.sh"
......@@ -26,3 +26,4 @@ services:
- POSTGRES_PASSWORD=adega
- POSTGRES_DB=adega
- POSTGRES_HOST=adega_db_1
- VERSION=DEVELOPMENT
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_USER=adega
- POSTGRES_PASSWORD=adega
- POSTGRES_DB=adega
web:
build:
context: .
dockerfile: ./docker_scripts/Dockerfile
command: bash "./docker_scripts/on_docker_init_production.sh"
volumes:
- .:/adega
# ports:
# - "8000:8000"
links:
- db
depends_on:
- db
environment:
- POSTGRES_USER=adega
- POSTGRES_PASSWORD=adega
- POSTGRES_DB=adega
- POSTGRES_HOST=adega_db_1
- VERSION=PRODUCTION
nginx:
restart: always
build: ./nginx/
ports:
- "8000:80"
depends_on:
- web
links:
- web:web
volumes:
- ./static:/adega/static
# This commands will be run inside of the container web
# If ANY of this commands fails (return != 0) the container will be down
bash ./docker_scripts/wait_for_postgres.sh
cd src
python manage.py makemigrations degree admission educator uploads course
python manage.py migrate
python manage.py collectstatic --noinput
#chmod 775 -R adega/static
#python manage.py runserver 0.0.0.0:8000
gunicorn adega.wsgi:application --workers 2 --timeout 600 -b :8000
......@@ -6,8 +6,7 @@ After=network.target
User=www-data
Group=www-data
WorkingDirectory=/var/www/adega/src
ExecStart=/var/www/adega/venv/bin/gunicorn --access-logfile - -k eventlet --workers 4 --bind unix:/var/www/adega/adega.sock adega.wsgi:application
ExecStart=/var/www/adega/venv/bin/gunicorn --access-logfile - -k eventlet --workers 2 --timeout 300 --bind unix:/var/www/adega/adega.sock adega.wsgi:application
[Install]
WantedBy=multi-user.target
......@@ -43,6 +43,9 @@ docker-fix:
docker-up:
docker-compose --project-directory . -f docker_scripts/docker-compose.yml -p adega up
docker-production:
docker-compose --project-directory . -f docker_scripts/docker-production.yml -p adega up
docker-remove-all:
docker rm adega_web_1 adega_db_1
docker rmi adega_web
......
FROM tutum/nginx
RUN rm /etc/nginx/sites-enabled/default
ADD sites-enabled/ /etc/nginx/sites-enabled
......@@ -31,7 +31,7 @@ server {
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
charset utf-8;
# certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
......@@ -75,7 +75,7 @@ server {
disable_symlinks off;
}
location /adega {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
......@@ -84,13 +84,13 @@ server {
proxy_pass http://adega/;
}
location /adega/static {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
root /var/www;
autoindex on;
try_files $uri $uri/ =404;
......@@ -102,5 +102,3 @@ server {
deny all;
}
}
server {
listen 80;
server_name localhost;
charset utf-8;
client_max_body_size 100M;
proxy_connect_timeout 600;
proxy_read_timeout 600;
uwsgi_read_timeout 600;
fastcgi_read_timeout 600;
keepalive_timeout 600;
location /static/ {
alias /adega/static/;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
......@@ -13,6 +13,9 @@ https://docs.djangoproject.com/en/1.9/ref/settings/
import os
from django.contrib import messages
import os
print()
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.join(BASE_DIR, '..')
......@@ -24,9 +27,9 @@ PROJECT_DIR = os.path.join(BASE_DIR, '..')
SECRET_KEY = 'e#-^aknk(5k)ej6rh#h$i(%h(m9)-j*lwrc_1dxnk=a@-mixlt'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
DEBUG = os.environ['VERSION'] == "DEVELOPMENT"
ALLOWED_HOSTS = []
ALLOWED_HOSTS = ["*"]
# Application definition
......@@ -43,11 +46,11 @@ INSTALLED_APPS = [
'adega',
'public',
'degree',
'educator',
'admission',
'course',
'student',
'degree',
'educator',
'admission',
'course',
'student',
'report_api',
'uploads'
]
......@@ -156,12 +159,14 @@ MESSAGE_TAGS = {
if not DEBUG:
FORCE_SCRIPT_NAME = '/adega/'
# FORCE_SCRIPT_NAME = '/adega/'
# STATIC_URL = '/static/'
STATIC_URL = '/static/'
STATIC_URL = '/adega/static/'
else:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(PROJECT_DIR, 'static')
MEDIA_URL = '/script/base/'
......
......@@ -4,8 +4,21 @@
{% block content%}
<h2>Dashboard</h2>
{% for degree in degrees %}
<a href="{% url 'degree:index' degree_id=degree.code %}">{{degree}} </a>
<div class="alert alert-warning" role="alert">
Os links a seguir exibirão os resumos (gráficos e taxas) dos <b>últimos relatórios
submetidos e processados</b> referentes aos cursos registrados no sistema. Caso não exista nenhum
relatório processado relacionado aos cursos, o sistema acusará um erro.
<a href="{% url 'uploads:SubmissionCreateView' %}">Submeter relatório</a></a>,
que podem ser gerenciados na página <a href="{% url 'uploads:SubmissionListView' %}"> Gerenciar relatórios</a>.
<br><br>
</div>
{% for degree in degrees_last_submissions %}
<a href="{% url 'degree:index' submission_id=degree.last_submission.id %}">{{degree.name}} ({{degree.code}})</a><br>
{% endfor %}
{% endblock content %}
......@@ -5,8 +5,8 @@
<div class="collapse navbar-collapse" id="navbartoggle">
<span class="navbar-brand mr-auto">
{% if degree != None %}
<a href="{% url 'degree:index' degree_id=degree.code%}">{{degree.name}} </a>
{% endif %}
<a href="{% url 'degree:index' submission_id=submission.id%}">{{degree.name}} </a>
{% endif %}
</span>
......@@ -19,7 +19,7 @@
{% if len(user.educator.degree.all()) > 1 %}
{% for degree in user.educator.degree.all() %}
<li><a href="{% url 'degree:index'
degree_id=degree.code%}">{{degree.name}} </a></li>
submission_id=submission.id %}">{{degree.name}} </a></li>
{% endfor %}
{% else %}
{% endif %}
......@@ -27,7 +27,10 @@
</li>
{% endcomment %}
<span class="navbar-text">
Relatório 2016/1
<a href="{% url 'dashboard' %}">Início</a>
</span>
<span class="navbar-text">
<a href="{% url 'uploads:SubmissionListView' %}">Relatórios</a>
</span>
&nbsp;&nbsp;&nbsp;
{% comment %}
......@@ -63,10 +66,10 @@
<ul class="nav navbar-nav hidden-md hidden-lg">
<li><a href="{% url 'student:index' degree_id=degree.code%}">Alunos</a></li>
<li><a href="{% url 'course:index' degree_id=degree.code%}">Turmas de Ingresso</a></li>
<li><a href="{% url 'student:index' submission_id=submission.id%}">Alunos</a></li>
<li><a href="{% url 'course:index' submission_id=submission.id%}">Turmas de Ingresso</a></li>
<li class="dropdown">
<a href="{% url 'course:index' degree_id=degree.code%}" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
<a href="{% url 'course:index' submission_id=submission.id%}" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
Disciplinas <span class="caret"></span></a>
<ul class="dropdown-menu" role="menu">
<li><a href="#">Lista Disciplinas</a></li>
......
<div class="sidebar left hidden-sm hidden-xs">
<li><a class="btn btn-primary text-left" href="{% url 'dashboard' %}">Gerenciar relatórios</a></li>
<li><a class="btn btn-primary text-left" href="{% url 'degree:index' degree_id=degree.code%}">Informações gerais do curso</a></li>
<li><a class="btn btn-primary text-left" href="{% url 'student:index' degree_id=degree.code%}">Alunos</a></li>
<li><a class="btn btn-primary text-left" href="{% url 'uploads:SubmissionListView' %}">Gerenciar relatórios</a></li>
<li><a class="btn btn-primary text-left" href="{% url 'degree:index' submission_id=submission.id%}">Informações gerais do curso</a></li>
<li><a class="btn btn-primary text-left" href="{% url 'student:index' submission_id=submission.id%}">Alunos</a></li>
<li style="display: block;">
<div>
<a class="btn btn-primary text-left" href="{% url 'course:index' degree_id=degree.code%}">Disciplinas</a>
<a class="btn btn-primary text-left" href="{% url 'course:index' submission_id=submission.id%}">Disciplinas</a>
{% comment %}
<a class="btn btn-primary text-left" href="course" class="drop" data-toggle="collapse" data-target="#side-disciplinas">
<span class="rotate"><i class="fa fa-angle-left"></i></span>
......@@ -21,7 +21,7 @@
<li><a class="btn btn-primary disabled text-left" href="#">Professores</a></li>
<li><a class="btn btn-primary disabled text-left" href="#">Turmas</a></li>
{% endcomment %}
<li><a class="btn btn-primary text-left" href="{% url 'admission:index' degree_id=degree.code%}">Turmas de Ingresso</a></li>
<li><a class="btn btn-primary text-left" href="{% url 'admission:index' submission_id=submission.id%}">Turmas de Ingresso</a></li>
{% comment %}
<li style="display: block;" >
<a href="#" data-toggle="collapse" data-target="#side-outros" style="display:flex" class="drop">
......
......@@ -4,22 +4,26 @@ from django.contrib import admin
from . import views
from django.views.generic import RedirectView
urlpatterns = [
url(r'^$', views.dashboard, name='dashboard'),
url(r'^$', RedirectView.as_view(url='/adega/')),
url(r'^adega/$', views.dashboard, name='dashboard'),
url(r'^adega/admission/(?P<submission_id>\w*)/', include('admission.urls', namespace='admission')),
url(r'^adega/course/(?P<submission_id>\w*)/', include('course.urls', namespace='course')),
url(r'^adega/submission/', include('uploads.urls', namespace='uploads')),
url(r'^uploads/', include('uploads.urls', namespace='uploads')),
url(r'^adega/student/(?P<submission_id>\w*)/', include('student.urls', namespace='student')),
url(r'^admission/(?P<degree_id>\w*)/', include('admission.urls', namespace='admission')),
url(r'^course/(?P<degree_id>\w*)/', include('course.urls', namespace='course')),
url(r'^student/(?P<degree_id>\w*)/', include('student.urls', namespace='student')),
url(r'^degree/(?P<degree_id>\w*)/', include('degree.urls', namespace='degree')),
url(r'^adega/degree/(?P<submission_id>\w*)/', include('degree.urls', namespace='degree')),
url(r'^public/', include('public.urls', namespace='public')),
url(r'^adega/public/', include('public.urls', namespace='public')),
url(r'^logout/$', views.logout, name='logout'),
url(r'^adega/logout/$', views.logout, name='logout'),
url(r'^admin/', admin.site.urls),
url(r'^adega/admin/', admin.site.urls),
]
from django.shortcuts import render, redirect
from django.contrib.auth.decorators import login_required
from django.contrib.auth import logout as process_logout
from uploads.models import Submission
@login_required
def dashboard(request):
degree = request.user.educator.degree.all()
degrees = request.user.educator.degree.all()
degrees_last_submissions = []
for degree in degrees:
last_submission = Submission.objects.filter(
degree=degree
).last()
degrees_last_submissions.append({
"name": degree.name,
"code": degree.code,
"last_submission": last_submission
})
return render(request, 'adega/dashboard.html', {'title': 'Dashboard',
"degrees":degree, "hide_navbar": True
"degrees_last_submissions":degrees_last_submissions, "hide_navbar": True
})
......
......@@ -24,7 +24,7 @@
{% for ti in listage_admissions %}
<tr>
<td>
<a href="{% url 'admission:detail' degree_id=degree.code ano=ti.ano semestre=ti.semestre %}">
<a href="{% url 'admission:detail' submission_id=submission.id ano=ti.ano semestre=ti.semestre %}">
{{ ti.ano }}/{{ ti.semestre }}
</a>
</td>
......
......@@ -5,11 +5,14 @@ from django.contrib import messages
from degree.models import Degree
from report_api.views import get_list_admission, get_admission_detail
from uploads.models import Submission
def detail(request, submission_id, ano, semestre):
submission_id = int(submission_id)
submission = Submission.objects.get(id=submission_id)
degree = submission.degree
def detail(request, degree_id, ano, semestre):
degree = Degree.objects.get(code=degree_id)
if not (degree in request.user.educator.degree.all()):
return redirect("adega:dashboard")
......@@ -31,16 +34,21 @@ def detail(request, degree_id, ano, semestre):
return render(request, 'admission/detail.html',{
"degree": degree,
"admission_info": admission_info
"admission_info": admission_info,
"submission": submission
})
def index(request, degree_id):
degree = Degree.objects.get(code=degree_id)
def index(request, submission_id):
submission_id = int(submission_id)
submission = Submission.objects.get(id=submission_id)
degree = submission.degree
if not (degree in request.user.educator.degree.all()):
return redirect("adega:dashboard")
return render(request, 'admission/index.html', {
"listage_admissions": get_list_admission(request.session, degree),
"degree": degree
"degree": degree,
"submission": submission
})
......@@ -21,7 +21,7 @@
{% for key, value in courses.items %}
<tr>
<td>
<a href="{% url 'course:detail' degree_id=degree.code codigo_disciplina=key %}">{{key}}</a>
<a href="{% url 'course:detail' submission_id=submission.id codigo_disciplina=key %}">{{key}}</a>
</td>
<td> {{value.name}} </td>
<td> {{value.nota.0 | floatformat:2}} &plusmn {{value.nota.1 | floatformat:2}} </td>
......
......@@ -5,11 +5,13 @@ from django.contrib import messages
from degree.models import Degree
from report_api.views import get_list_courses, get_course_detail
from uploads.models import Submission
def detail(request, degree_id, codigo_disciplina):
degree = Degree.objects.get(code=degree_id)
def detail(request, submission_id, codigo_disciplina):
submission_id = int(submission_id)
submission = Submission.objects.get(id=submission_id)
degree = submission.degree
if not (degree in request.user.educator.degree.all()):
return redirect("adega:dashboard")
......@@ -18,13 +20,17 @@ def detail(request, degree_id, codigo_disciplina):
return render(request, 'course/detail.html',{
"analysis_result": course_detail,
"degree": degree,
"submission": submission,
"codigo_disciplina": codigo_disciplina,
"nome_disciplina": course_detail["disciplina_nome"]
})
def index(request, degree_id):
degree = Degree.objects.get(code=degree_id)
def index(request, submission_id):
submission_id = int(submission_id)
submission = Submission.objects.get(id=submission_id)
degree = submission.degree
if not (degree in request.user.educator.degree.all()):
return redirect("adega:dashboard")
......@@ -36,5 +42,5 @@ def index(request, degree_id):
return render(request, 'course/index.html', {
"courses": courses_list,
"degree": degree
"submission": submission
})
......@@ -9,13 +9,22 @@ import json
@login_required
def index(request,degree_id):
degree = Degree.objects.get(code=degree_id)
def index(request, submission_id):
submission_id = int(submission_id)
submission = Submission.objects.get(id=submission_id)
degree = submission.degree
if not (degree in request.user.educator.degree.all()):
return redirect("adega:dashboard")
degree_data = get_degree_information(request.session,degree)
return render(request,"degree/index.html",{"degree":degree,"degree_data":degree_data})
degree_data = get_degree_information(request.session,degree, submission_id=submission_id)
return render(request,"degree/index.html",{
"submission":submission,
"degree": degree,
"degree_data":degree_data
})
#class Views(View):
# template_name = "index.html"
......
......@@ -2,8 +2,10 @@ from django.db import models
from django.contrib.auth.models import User
from degree.models import Degree
class Educator(models.Model):
user = models.OneToOneField(User)
degree = models.ManyToManyField(Degree)
user = models.OneToOneField(User, related_name='educator')
degree = models.ManyToManyField(Degree)
def __str__(self):
return "{}".format(self.user.username)
return "{}".format(self.user.username)
......@@ -4,4 +4,4 @@ from public import views
urlpatterns = [
url(r'^$', views.index, name="index"),
]
\ No newline at end of file
]
......@@ -2,34 +2,36 @@ from degree.models import Degree
from uploads.models import Submission
import json
def get_data(session, degree, data_name):
if "submission" in session:
submission = session["submission"]
def get_data(session, degree, data_name, submission_id=None):
if(submission_id):
submission = Submission.objects.filter(id=submission_id).last()
elif "submission" in session:
submission = session["submission"]
else:
submission = Submission.objects.filter(degree=degree).last()
submission = Submission.objects.filter(degree=degree).last()
path_data = submission.path() + "/" + data_name
path_data = submission.path() + "/" + data_name
with open(path_data) as data_f:
data = json.load(data_f)
data = json.load(data_f)
return data
def get_degree_information(session, degree):
return get_data(session,degree,"degree.json")
def get_degree_information(session, degree, submission_id=None):
return get_data(session,degree,"degree.json", submission_id=submission_id)
def get_list_admission(session, degree):
return get_data(session,degree,"admissions/lista_turma_ingresso.json")
def get_list_admission(session, degree, submission_id=None):
return get_data(session,degree,"admissions/lista_turma_ingresso.json", submission_id=submission_id)
def get_admission_detail(session, degree, year, semester):
return get_data(session,degree,"admissions/"+year+"/"+semester+".json")
def get_admission_detail(session, degree, year, semester, submission_id=None):
return get_data(session,degree,"admissions/"+year+"/"+semester+".json", submission_id=submission_id)
def get_list_courses(session, degree):
return get_data(session,degree,"courses/disciplinas.json")
def get_list_courses(session, degree, submission_id=None):
return get_data(session,degree,"courses/disciplinas.json", submission_id=submission_id)
def get_course_detail(session, degree, course_id):
return get_data(session,degree,"courses/"+course_id+".json")
def get_course_detail(session, degree, course_id, submission_id=None):
return get_data(session,degree,"courses/"+course_id+".json", submission_id=submission_id)
def get_list_students(session, degree, list_name):
return get_data(session,degree,"students/list/"+list_name+".json")
def get_list_students(session, degree, list_name, submission_id=None):
return get_data(session,degree,"students/list/"+list_name+".json", submission_id=submission_id)
def get_student_detail(session, degree, student_id):
return get_data(session,degree,"students/"+student_id+".json")
def get_student_detail(session, degree, student_id, submission_id=None):
return get_data(session,degree,"students/"+student_id+".json", submission_id=submission_id)
......@@ -173,7 +173,6 @@ def generate_admission_data(path, df):
("alunos_evadidos", evasion_count["alunos_evadidos"]),
("outras_formas_evasao", evasion_count["outras_formas_evasao"])
]
print(a.build_cache_evasion_count())
# cria um dicionario com as analises para cada turma
turmas = defaultdict(dict)
for a in analises:
......
......@@ -6,20 +6,32 @@ from script.build_cache import build_cache
from datetime import timedelta
def analyze(submission):
start_time = time.clock()
start_time_exec = time.time()
path = submission.path()
dataframe = load_dataframes(path)
build_cache(dataframe, path)
def analyze(submission, debug=True):
print(submission.path())
submission.set_done(round(time.clock() - start_time))
cpu_time = timedelta(seconds=round(time.clock() - start_time))
run_time = timedelta(seconds=round(time.time() - start_time_exec))
print("--- Tempo de CPU: {} ---".format(cpu_time))
print("--- Tempo total: {} ---".format(run_time))
start_time = time.clock()
start_time_exec = time.time()
try:
path = submission.path()
dataframe = load_dataframes(path)
build_cache(dataframe, path)
submission.set_done(round(time.clock() - start_time))
cpu_time = timedelta(seconds=round(time.clock() - start_time))
run_time = timedelta(seconds=round(time.time() - start_time_exec))
if(debug):
print("--- Tempo de CPU: {} ---".format(cpu_time))
print("--- Tempo total: {} ---".format(run_time))
except:
if(debug):
print("Error on submission analysis:",ValueError)
submission.set_fail(round(time.clock() - start_time))
def main():
......
......@@ -87,7 +87,7 @@
<tbody>
{% for at in analysis_result.aluno_turmas %}
<tr>