Tutorials
=========


Learn through practical examples. The following tutorials cover various scenarios, from text embedding for vector search to building custom web corpora and generating word frequency lists.


.. toctree::
   :maxdepth: 2

   tutorial0
   tutorial1
   tutorial2
   tutorial-epsilla
   tutorial-dwds



Blog posts
^^^^^^^^^^

- `Extracting the main text content from web pages using Python <https://adrien.barbaresi.eu/blog/trafilatura-main-text-content-python.html>`_
- `Validating TEI-XML documents with Python <https://adrien.barbaresi.eu/blog/validating-tei-xml-python.html>`_
- `Evaluating scraping and text extraction tools for Python <https://adrien.barbaresi.eu/blog/evaluating-text-extraction-python.html>`_
- `Filtering links to gather texts on the web <https://adrien.barbaresi.eu/blog/link-filtering-courlan-python.html>`_
- `Using sitemaps to crawl websites on the command-line <https://adrien.barbaresi.eu/blog/using-sitemaps-crawl-websites.html>`_
- `Using RSS and Atom feeds to collect web pages with Python <https://adrien.barbaresi.eu/blog/using-feeds-text-extraction-python.html>`_
- `Web scraping with R: Text and metadata extraction <https://adrien.barbaresi.eu/blog/web-scraping-text-metadata-r.html>`_
- `Web scraping with Trafilatura just got faster <https://adrien.barbaresi.eu/blog/web-scraping-trafilatura-faster.html>`_


Videos
^^^^^^

Youtube playlist with video tutorials in several languages
    `Web scraping how-tos and tutorials <https://www.youtube.com/watch?v=8GkiOM17t0Q&list=PL-pKWbySIRGMgxXQOtGIz1-nbfYLvqrci>`_.


.. raw:: html

    <iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/rEOoItpzlVw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>



External resources
^^^^^^^^^^^^^^^^^^

- `GLAM-Workbench <https://github.com/GLAM-Workbench>`_
   - `Harvesting collections of text from archived web pages <https://github.com/GLAM-Workbench/web-archives/blob/master/getting_text_from_web_pages.ipynb>`_
   - `Compare two versions of an archived web page <https://github.com/GLAM-Workbench/web-archives/blob/master/show_diffs.ipynb>`_


- `User Ethics & Legal Concerns <https://melaniewalsh.github.io/Intro-Cultural-Analytics/Data-Collection/User-Ethics-Legal-Concerns.html>`_
- `Download von Web-Daten <https://www.bubenhofer.com/korpuslinguistik/kurs/index.php?id=eigenes_wwwdownload.html>`_ & `Daten aufbereiten und verwalten <https://www.bubenhofer.com/korpuslinguistik/kurs/index.php?id=eigenes_aufbereitenXML.html>`_ (Tutorials in German by Noah Bubenhofer)