mirror of
https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git
synced 2025-07-31 15:55:18 +00:00
Some fixes and improve jinja2 template performance by reducing include statements
This commit is contained in:
Patrick Jentsch
@@ -41,10 +41,58 @@
|
||||
<div class="job-list" data-user-id="{{ current_user.hashid }}"></div>
|
||||
</div>
|
||||
<div class="card-action right-align">
|
||||
<p><a href="#data-processing-and-analysis-modal" class="btn modal-trigger waves-effect waves-light">Create job<i class="material-icons right">add</i></a></p>
|
||||
<p><a data-target="dashboard-create-job-dropdown-content" class="btn waves-effect waves-light dropdown-trigger no-autoinit" id="dashboard-create-job-dropdown-trigger">Create job<i class="material-icons right">add</i></a></p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock page_content %}
|
||||
|
||||
|
||||
{% block dropdowns %}
|
||||
{{ super() }}
|
||||
<ul class="dropdown-content" id="dashboard-create-job-dropdown-content">
|
||||
<li>
|
||||
<a href="{{ url_for('services.file_setup_pipeline') }}">
|
||||
<i class="nopaque-icons service-icons service-color-text text-darken" data-service="file-setup-pipeline"></i>
|
||||
File Setup Pipeline
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="{{ url_for('services.tesseract_ocr_pipeline') }}">
|
||||
<i class="nopaque-icons service-icons service-color-text text-darken" data-service="tesseract-ocr-pipeline"></i>
|
||||
Tesseract OCR Pipeline
|
||||
</a>
|
||||
</li>
|
||||
{% if config.NOPAQUE_TRANSKRIBUS_ENABLED %}
|
||||
<li>
|
||||
<a href="{{ url_for('services.transkribus_htr_pipeline') }}">
|
||||
<i class="nopaque-icons service-icons service-color-text text-darken" data-service="transkribus-htr-pipeline"></i>
|
||||
Transkribus HTR Pipeline
|
||||
</a>
|
||||
</li>
|
||||
{% endif %}
|
||||
<li>
|
||||
<a href="{{ url_for('services.spacy_nlp_pipeline') }}">
|
||||
<i class="nopaque-icons service-icons service-color-text text-darken" data-service="spacy-nlp-pipeline"></i>
|
||||
SpaCy NLP Pipeline
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
{% endblock dropdowns %}
|
||||
|
||||
|
||||
{% block scripts %}
|
||||
{{ super() }}
|
||||
<script>
|
||||
M.Dropdown.init(
|
||||
document.querySelector('#dashboard-create-job-dropdown-trigger'),
|
||||
{
|
||||
constrainWidth: false,
|
||||
container: document.querySelector('#dropdowns'),
|
||||
coverTrigger: false
|
||||
}
|
||||
);
|
||||
</script>
|
||||
{% endblock scripts %}
|
||||
|
||||
@@ -20,8 +20,8 @@
|
||||
<h6 style="font-weight: 300;">Changes to the Query Builder</h6>
|
||||
<p>
|
||||
The Query Builder has undergone changes to make it more intuitive to use and is now the standard option for creating queries.
|
||||
Individual elements of a query can now be easily modified and edited by clicking on them.
|
||||
An input marker shows your position in the inquiry and where new elements will be added. This and all other elements can be moved around via drag and drop.
|
||||
Individual elements of a query can now be easily modified and edited by clicking on them.
|
||||
An input marker shows your position in the inquiry and where new elements will be added. This and all other elements can be moved around via drag and drop.
|
||||
A new toggle button enables users to easily switch between the Query Builder and Expert Mode if they prefer to work with the plain Corpus Query Language (CQL) instead. This can be done in the middle of an existing query – existing chips will be “translated” into CQL.
|
||||
This also works the other way around – if you want to switch back, your query in CQL wll be parsed into chips.
|
||||
More details and instructions on how to use the new Query Builder can be found in the manual.
|
||||
@@ -29,9 +29,9 @@
|
||||
<br>
|
||||
<h6 style="font-weight: 300;">Community Update</h6>
|
||||
<p>
|
||||
The most extensive changes to nopaque have taken place in the Social Area. We want nopaque to be a platform where researchers can connect with each other, so we’ve added some more features to make this possible.
|
||||
The most extensive changes to nopaque have taken place in the Social Area. We want nopaque to be a platform where researchers can connect with each other, so we’ve added some more features to make this possible.
|
||||
Users can now update their personal profiles to be publicly visible to others on nopaque, including a short “About me” section and options to share your website, organization, location, and add an avatar that others can see.
|
||||
It is also possible to share corpora with other researchers via share links, access invitations, or by setting corpus visibility to Public. Other users can only see the meta data of public corpora – further access can be granted upon request.
|
||||
It is also possible to share corpora with other researchers via share links, access invitations, or by setting corpus visibility to Public. Other users can only see the meta data of public corpora – further access can be granted upon request.
|
||||
The extent of access to these shared corpora is managed by assigning the roles of Viewer, Contributor, and Administrator. Viewers may only download the files. Contributors can download and edit files and their metadata as well as analyze and build the corpus. Administrators can manage users, followers and visibility, in addition to all of the above.
|
||||
</p>
|
||||
<br>
|
||||
@@ -47,21 +47,21 @@
|
||||
<br>
|
||||
<p>Hey users,</p>
|
||||
<p>
|
||||
we wanted to give you some news on updates we’re making to nopaque.
|
||||
Since we want to make it easier for users to grasp and work with different elements of their data,
|
||||
we’ve been working on adding some visualization features into the Corpus Analysis service. Currently, the two main modules,
|
||||
we wanted to give you some news on updates we’re making to nopaque.
|
||||
Since we want to make it easier for users to grasp and work with different elements of their data,
|
||||
we’ve been working on adding some visualization features into the Corpus Analysis service. Currently, the two main modules,
|
||||
“Reader” and “Concordance” have been expanded with an additional “Static Visualizations” module, but there’s more to come!
|
||||
</p>
|
||||
<p>
|
||||
With the Static Visualizations module, it’s now possible to view information
|
||||
about your corpus, such as the number of (unique) tokens, sentences, lemmata,
|
||||
corresponding information on individual texts, the distribution of these elements
|
||||
within your corpus, as well as searchable lists of word frequencies with stopwords
|
||||
With the Static Visualizations module, it’s now possible to view information
|
||||
about your corpus, such as the number of (unique) tokens, sentences, lemmata,
|
||||
corresponding information on individual texts, the distribution of these elements
|
||||
within your corpus, as well as searchable lists of word frequencies with stopwords
|
||||
that can be preset and modified. In the future, this area will be extended with more advanced visualization options.
|
||||
</p>
|
||||
<p>
|
||||
We’ll keep you posted about further visualization updates. Until then, we hope the latest update improves
|
||||
your research experience with nopaque. And as always, if you have any ideas for nopaque or need assistance,
|
||||
We’ll keep you posted about further visualization updates. Until then, we hope the latest update improves
|
||||
your research experience with nopaque. And as always, if you have any ideas for nopaque or need assistance,
|
||||
don’t hesitate to contact us!
|
||||
</p>
|
||||
<br>
|
||||
@@ -78,7 +78,7 @@
|
||||
<p>Dear users,</p>
|
||||
<p>
|
||||
users can now upload their own language models into nopaque. This is useful for working with different languages that are not available as standard in nopaque or if a user wants to work with a language model they have developed themselves. Tesseract models can be uploaded in .traineddata format; spaCy models can be uploaded in tar.gz format. We are also working on the option to upload models in .whl format in the future.
|
||||
Uploaded models can be found in the model list of the corresponding service and can be used immediately. Models can also be made public if you have a role of Contributor in nopaque.
|
||||
Uploaded models can be found in the model list of the corresponding service and can be used immediately. Models can also be made public if you have a role of Contributor in nopaque.
|
||||
</p>
|
||||
<br>
|
||||
<p><b>Please note:</b> The Contributor role must be requested from the nopaque admins if you would like to make a model public for all users.</p>
|
||||
@@ -86,7 +86,7 @@
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="col s12">
|
||||
<div class="card" id="news-post-april-2022">
|
||||
<div class="card-content">
|
||||
@@ -95,62 +95,62 @@
|
||||
<br>
|
||||
<p>Hello everyone,</p>
|
||||
<p>
|
||||
in April 2022, we released an update improving many elements of nopaque. We rewrote a lot of our code,
|
||||
including a significant reworking of our backend code for more efficient use of our servers.
|
||||
in April 2022, we released an update improving many elements of nopaque. We rewrote a lot of our code,
|
||||
including a significant reworking of our backend code for more efficient use of our servers.
|
||||
We integrated a new service, updated the existing ones, and made some minor design improvements.
|
||||
</p>
|
||||
<br>
|
||||
<h6 style="font-weight: 300;">Database Cleanup</h6>
|
||||
<p>
|
||||
We may be a bit late with our spring cleaning, but we’ve tidied up our
|
||||
database system and deleted old, empty corpora, unconfirmed user accounts and
|
||||
We may be a bit late with our spring cleaning, but we’ve tidied up our
|
||||
database system and deleted old, empty corpora, unconfirmed user accounts and
|
||||
unnecessary data fields.
|
||||
</p>
|
||||
<h6 style="font-weight: 300;">What's new?</h6>
|
||||
<p>
|
||||
By partnering with Transkribus, we’ve reached one of our long-term goals: to integrate a
|
||||
Handwritten Text Recognition (HTR) service into nopaque. The Transkribus HTR Pipeline service is implemented as a
|
||||
kind of proxied service where the work is split between us and Transkribus.
|
||||
By partnering with Transkribus, we’ve reached one of our long-term goals: to integrate a
|
||||
Handwritten Text Recognition (HTR) service into nopaque. The Transkribus HTR Pipeline service is implemented as a
|
||||
kind of proxied service where the work is split between us and Transkribus.
|
||||
That means we do the preprocessing, storage and postprocessing, while Transkribus handles the HTR itself.
|
||||
</p>
|
||||
<p>
|
||||
One change we needed to make in the background was to fix our performance issues.
|
||||
While implementing the Transkribus HTR Pipeline service, we saw optimization potential
|
||||
in different steps of our processing routine. These optimizations are now also available
|
||||
in our Tesseract OCR Pipeline service and result in speeds that are about four times faster
|
||||
than before. We’re now finished with the major optimizations, but there could be more soon,
|
||||
One change we needed to make in the background was to fix our performance issues.
|
||||
While implementing the Transkribus HTR Pipeline service, we saw optimization potential
|
||||
in different steps of our processing routine. These optimizations are now also available
|
||||
in our Tesseract OCR Pipeline service and result in speeds that are about four times faster
|
||||
than before. We’re now finished with the major optimizations, but there could be more soon,
|
||||
so stay tuned!
|
||||
</p>
|
||||
<p>
|
||||
Next, we reorganized our Corpus Analysis code. It was a bit messy, but after a complete rewrite,
|
||||
we are now able to query a corpus without long loading times and with better error handling,
|
||||
making the user experience much more stable. The Corpus Analysis service is now modularized and comes with two modules
|
||||
Next, we reorganized our Corpus Analysis code. It was a bit messy, but after a complete rewrite,
|
||||
we are now able to query a corpus without long loading times and with better error handling,
|
||||
making the user experience much more stable. The Corpus Analysis service is now modularized and comes with two modules
|
||||
that recreate and extend the functionality of the old service.
|
||||
</p>
|
||||
<p>
|
||||
The Query Result viewer had to be temporarily disabled, as the code was based on the old Corpus Analysis service.
|
||||
The Query Result viewer had to be temporarily disabled, as the code was based on the old Corpus Analysis service.
|
||||
It will be reintegrated as a module to the Corpus Analysis.
|
||||
</p>
|
||||
<p>
|
||||
The spaCy NLP Pipeline service was also taken care of with some smaller updates. This is important preliminary work
|
||||
for support of more models/languages missing the full set of linguistic features (lemma, ner, pos, simple_pos).
|
||||
It still needs some testing and adjustments but will be ready soon!
|
||||
The spaCy NLP Pipeline service was also taken care of with some smaller updates. This is important preliminary work
|
||||
for support of more models/languages missing the full set of linguistic features (lemma, ner, pos, simple_pos).
|
||||
It still needs some testing and adjustments but will be ready soon!
|
||||
</p>
|
||||
<p>
|
||||
Last, but not least, we made some design changes. Now, you can find color in places that were previously in black and white.
|
||||
Nothing big, but the new colors can aid in identifying resources more efficiently.
|
||||
Last, but not least, we made some design changes. Now, you can find color in places that were previously in black and white.
|
||||
Nothing big, but the new colors can aid in identifying resources more efficiently.
|
||||
</p>
|
||||
<h6 style="font-weight: 300;">Where is my job data?</h6>
|
||||
<p>
|
||||
We reached our storage limit at the beginning of the year.
|
||||
At this time, some users may have noticed system instability.
|
||||
Fortunately, we found a solution that avoided data loss by deleting some
|
||||
non-nopaque related data in our system (yes, <a href="https://www.uni-bielefeld.de/fakultaeten/geschichtswissenschaft/abteilung/arbeitsbereiche/digital-history/">we also do things other than nopaque</a>).
|
||||
To avoid facing the same problem again, we had to find a long-term solution.
|
||||
In the end, this involved the deletion of all previous job data with this update and,
|
||||
going forward, only keeping new job data for three months after job creation
|
||||
(<b>important note:</b> corpora are not affected). All job data created prior to this
|
||||
update has been backed up for you. Feel free to contact us at <a href="mailto:nopaque@uni-bielefeld.de">nopaque@uni-bielefeld.de</a>
|
||||
We reached our storage limit at the beginning of the year.
|
||||
At this time, some users may have noticed system instability.
|
||||
Fortunately, we found a solution that avoided data loss by deleting some
|
||||
non-nopaque related data in our system (yes, <a href="https://www.uni-bielefeld.de/fakultaeten/geschichtswissenschaft/abteilung/arbeitsbereiche/digital-history/">we also do things other than nopaque</a>).
|
||||
To avoid facing the same problem again, we had to find a long-term solution.
|
||||
In the end, this involved the deletion of all previous job data with this update and,
|
||||
going forward, only keeping new job data for three months after job creation
|
||||
(<b>important note:</b> corpora are not affected). All job data created prior to this
|
||||
update has been backed up for you. Feel free to contact us at <a href="mailto:nopaque@uni-bielefeld.de">nopaque@uni-bielefeld.de</a>
|
||||
if you would like to get this data back.
|
||||
</p>
|
||||
<br>
|
||||
@@ -173,10 +173,10 @@
|
||||
<li>Corpus analysis, which makes use of CQP Query Language to search through text corpora with the aid of metadata and Natural Language Processing tags.</li>
|
||||
</ul>
|
||||
<p>
|
||||
Nopaque was created based on our experiences working with other subprojects and a Prototyp user study in the
|
||||
first phase of funding. The platform is open source under the terms of the MIT license (<a href="https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque">https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque</a>).
|
||||
Language support and functions are currently limited – extensions can be requested by sending an email to <a href="mailto:nopaque@uni-bielefeld.de">nopaque@uni-bielefeld.de</a>.
|
||||
Because we are still in the beta phase, some bugs are to be expected. If you encounter any problems, please let us know!
|
||||
Nopaque was created based on our experiences working with other subprojects and a Prototyp user study in the
|
||||
first phase of funding. The platform is open source under the terms of the MIT license (<a href="https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque">https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque</a>).
|
||||
Language support and functions are currently limited – extensions can be requested by sending an email to <a href="mailto:nopaque@uni-bielefeld.de">nopaque@uni-bielefeld.de</a>.
|
||||
Because we are still in the beta phase, some bugs are to be expected. If you encounter any problems, please let us know!
|
||||
We are thankful for all feedback we receive.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<div class="col s12">
|
||||
<h1 id="title">{{ title }}</h1>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="col s12">
|
||||
<div id="aggregated-news"></div>
|
||||
<div class="card" id="april-2022-update">
|
||||
|
||||
@@ -58,7 +58,7 @@
|
||||
<span class="card-title">§ 2 General information on data processing and its purpose</span>
|
||||
<p>We process the personal data of our users only to the extent necessary to provide a functioning website and its functionalities. Collecting this information enables us to better diagnose problems with the application, provide support more effectively as well as ensure the continuous functionality of the service.</p>
|
||||
<p> The following (personal) data is collected and stored within the system:</p>
|
||||
|
||||
|
||||
<h6>Master Data</h6>
|
||||
<p>Within the scope of user authentication, the following personal data is collected and processed: </p>
|
||||
<ul class="browser-default">
|
||||
@@ -66,7 +66,7 @@
|
||||
<li>E-Mail</li>
|
||||
</ul>
|
||||
<p>The registration of users is required for the provision of access to services within NOPAQUE. The freely selectable username and the corresponding email address is used to persistently identify you in NOPAQUE. The provided email address might be used to contact you in case we noticed some malfunction, to announce maintenance, or to spread important information regarding Nopaque. If you reset your password, Nopaque will use your email address to send you reset instructions via email. Nopaque does not show your email address and the username to other Nopaque users per default – except, if user gave the permission to do so, according to <b>§ 7 paragraph 1</b> of the General Terms of Use for the use of NOPAQUE.</p>
|
||||
|
||||
|
||||
<h6>Protocol and administrative data</h6>
|
||||
<p>In general, when a website is visited, for technical reasons information is automatically sent from the browser to the server and stored there in access protocols. When using a web application, additional protocol data is also generated, which is necessary for tracking technical errors. This information includes:</p>
|
||||
<ul class="browser-default">
|
||||
@@ -96,15 +96,15 @@
|
||||
<li><b>remember_token</b>: Login script with ‘remember me’ feature allowing the user to preserve their logged in status. When the user checks the Remember Me option, then the logged in status is serialized in the session and stored in cookies in an encrypted way.</li>
|
||||
</ul>
|
||||
<p>Cookies collected by NOPAQUE do not collect personal information of the users.</p>
|
||||
|
||||
|
||||
<h6>Content Data</h6>
|
||||
<p>The content data includes all data that is entered or created by users themselves in the system. This data is listed here because it is assigned to individual authors and may contain personal data. This may include: <b>uploaded files, images, text documents, other media files</b> and <b>(interim) results</b> after data processing and computations. Please note that files and scans submitted to NOPAQUE are safely stored on the NOPAQUE server in order to allow persistent access during a work session and between work sessions.</p>
|
||||
<br>
|
||||
<p>According to <b>§ 4 paragraph 1 - 3</b> of the General Terms of Use for the use of NOPAQUE at Bielefeld University, the users themselves are responsible for the content they upload and must comply with the legal provisions of data protection and copyright law. This includes in particular the deletion of personal data that may no longer be processed.</p>
|
||||
|
||||
|
||||
<h6>User-added Information (optional)</h6>
|
||||
<p>NOPAQUE also stores optionally user-added personal information, like user’s profile information (full name, affiliation) and user’s added profile photo (avatar).</p>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user