Python 抓取网页

分享于2022年07月17日 python selenium web web-scraping 问答
【问题标题】:Python 抓取网页(Python Scraping Web page)
【发布时间】:2022-01-27 18:52:13
【问题描述】:

我试图从这个示例链接中删除所有“a”标记占位符值: https://www.fundoodata.com/companies-in/list-of-apparel-stores-companies-in-india-i239

我只需要复制“a”标签的名称并需要将其保存到 csv 文件中

我是初学者,谁能帮帮我,下面是我的错误代码:

# importing the modules
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import pandas as pd
import time
import os


link = "https://www.fundoodata.com/companies-in/list-of-apparel-stores-companies-in-india-i239"




# instantiating empty lists
nameList = []


for i in range(1) :
    driver = webdriver.Chrome(ChromeDriverManager().install())

 
    # fetching all the store details
    storeDetails = driver.find_elements_by_class_name('search-result')
 

 
    # iterating the storeDetails
    for j in range(len(storeDetails)):
         
        # fetching the name, address and contact for each entry
        name = storeDetails[j].find_element_by_class_name('heading').text
    
         
        myList = []
         
        nameList.append(name)


    driver.close()
 
     
# initialize data of lists.
data = {'Company Name': nameList,}
 
# Create DataFrame
df = pd.DataFrame(data)
print(df)
 
# Save Data as .csv
df.to_csv("D:\xxx\xxx\xx\xxx\demo.csv", mode='w+', header = False)

  • 那么,您当前的代码有什么问题?什么不起作用?您看到了什么错误,在哪里?
  • 这段代码完全错误
  • 所以,你的问题应该更加集中。我们不是免费的编码服务,我们只能帮助解决一些特定的问题

【解决方案1】:

首先这里有很多问题:

## to open webpage
driver.get(link)

其次,您根本不需要第一个 for 循环。最后:

from selenium.webdriver.common.by import By
## find the a tags inside the search results
storedetails = driver.find_elements(By.CSS_SELECTOR, 'div.heading a')
## iterating over elements list
for name in storedetails:
    ## appending the a tag text
    nameList.append(name.text)

我希望这会有所帮助! :)