Welcome to pydomiporta’s documentation!

Introduction

pydomiporta supplies two methods that can be used to scrape data from Domiporta website

Scraping category data

This method scrapes available offer urls from Domiporta search results with parameters .. autofunction:: domiporta.category.get_category

The function above can be used like this:

filters = {'Price.From': 1500, 'Pietro.To': 3}
offer_urls = domiporta.category.get_category(url=None, 'Mieszkania', 'wynajme', 'Pomorskie', 'Gdańsk', filters)

The code above will put a list of urls containing all apartments found in the given category into the offers_url variable

Scraping offer data

This method scrapes details of offer .. autofunction:: domiporta.offer.get_offer_data

The function above can be used like this:

details = domiporta.offer.get_offer_data(url)

the code above will create dictionary with details of offer from given url

Category methods

domiporta.category.get_category(url=None, category='nieruchomosci', transaction_type='wszystkie', voivodeship=None, city=None, street=None, filters=None)[source]

Parses available offer urls from given category search page

Parameters:
  • url (str, None) – Url to search web page
  • category – Type of property of interest (Mieszkanie/Dom/Garaż/Działka)
  • transaction_type – Type of transaction
  • voivodeship – Voivodeship
  • city – City
  • street – Street
  • filters – Dictionary with additional filters

:type category:str, None :type transaction_type: str, None :type voivodeship: str, None :type city: str, None :type street: str, None :type filters: dict, None :return: List of urls of all offers for given parameters :rtype: list

domiporta.category.get_offers_from_category(url)[source]

Parses available offer urls from given category from given page

Parameters:url (str) – Defined url for Domiporta page with offers
Returns:List of urls from given page
Return type:list

Offer methods

domiporta.offer.get_gps_data(content)[source]

Parse latitude and longitude

Parameters:content – raw html
Returns:list with geographical coordinates or None if can’t find
Return type:list
domiporta.offer.get_meta_data(markup)[source]

Parse meta data

Parameters:markup – raw html
Returns:dictionary with data
Return type:dict
domiporta.offer.get_offer_data(url)[source]

Parse details about given offer

Parameters:url (str) – Url to offer web page
Returns:Details about given offer
Return type:dict

Utils methods

domiporta.utils.encode_text_to_html(text)[source]

Change text to lower cases, gets rid of polish characters replacing them with simplified version, replaces spaces with dashes

Parameters:text (str) – text to encode
Returns:encoded text which can be used in url
Return type:str
domiporta.utils.get_content_from_source(url)[source]

Connects with given url

If environmental variable DEBUG is True it will cache response for url in /var/temp directory

Parameters:url (str) – Website url
Returns:Response for requested url
domiporta.utils.get_url(category='nieruchomosci', transaction_type='wszystkie', voivodeship=None, city=None, street=None, filters=None)[source]

Create url to Domiporta search web page with given parameters and filters

Parameters:
  • category – Type of property of interest (Mieszkanie/Dom/Garaż/Działka)
  • transaction_type – Type of transaction
  • voivodeship – Voivodeship
  • city – City
  • street – Street
  • filters – Dictionary with additional filters

:type category:str, None :type transaction_type: str, None :type voivodeship: str, None :type city: str, None :type street: str, None :type filters: dict, None :return: Url to Domiporta search web page :rtype: str

Indices and tables