Web Scraping in Python

Anonymous User 1378 13 Jul 2018

How to Implement web Scraping in Python with BeautifulSoup ?

Anonymous User

Other

I am a content writter !

1 Answers

Prakash nidhi Verma 13 Jul 2018

Implementing Web Scraping in Python with BeautifulSoup:

Use the API of the website. example: Facebook has the Facebook Graph API which

- allows retrieval of data posted on Facebook.

- HTML of the webpage information/data will access.

Installing the required third-party libraries :

pip install requests 
pip install html5lib

pip install bs4

Accessing the HTML content from webpage :

import requests 
URL = "https://mindstick.com/forum/"

r = requests.get(URL)

print(r.content)

Parsing the HTML content :

#This will not run on online IDE 
import requests

from bs4 import BeautifulSoup

URL = "http://www.values.com/inspirational-quotes"

r = requests.get(URL)

soup = BeautifulSoup(r.content, 'html5lib')

print(soup.prettify())

soup = BeautifulSoup(r.content, 'html5lib')

table = soup.find('div', attrs = {'id':'container'})

quote['theme'] = row.h5.text

quote['url'] = row.a['href']

forum

Web Scraping in Python

Can you answer this question?

1 Answers