[Solved] Python code to authenticate to website, navigate through links and download files

by


After 5 days of research, I found what I wanted. Your urlLogin and urlAuth could be same, its totally depends on what action taken on Login button or form action. I used crome inspect option to findout the actual GET or POST request used on the portal.

Here is the answer of my own question–>

import requests

urlLogin = 'https://example.com/jsp/login.jsp'
urlAuth="https://example.com/CheckLoginServlet"
urlBd = 'https://example.com/jsp/batchdownload.jsp'
payload = {
    "username": "username",
    "password": "password"
}

# Session will be closed at the end of with block
with requests.Session() as s:
    s.get(urlLogin)
    headers = s.cookies.get_dict()
    print(f"Session cookies {headers}")
    r1 = s.post(urlAuth, data=payload, headers=headers)
    print(f'MainFrame text:::: {r1.status_code}')  #200

    r2 = s.post(urlBd, data=payload)
    print(f'MainFrame text:::: {r2.status_code}')  #200
    print(f'MainFrame text:::: {r2.text}')  #page source

    # 3. Again cookies will be used through session to access batch download page
    r2 = s.post(config['access-url'])
    print(f'Batch Download status:::: {r2.status_code}')  #200
    source_code = r2.text
    # print(f'Batch Download source:::: {source_code}')

1

solved Python code to authenticate to website, navigate through links and download files