beautifulsoup - Extracting data from a web page using BS4 in Python -

-

i trying extract data site: http://www.afl.com.au/fixture

in way such have dictionary having date key , "preview" links values in list, like

dict = {saturday, june 07: ["preview url-1, "preview url-2","preview url-3","preview url-4"]}

please me it, have used code below:

def extractdata():     ldateinfomatchcase = false #     ldateinfomatchcase = []     global gdict     row in table_for_players.findall("tr"):         ldaterowindex in row.findall("th", {"colspan" : "4"}):             ldatelist.append(ldaterowindex.text)      print ldatelist     index in ldatelist:         #print index         lpreviewlinklist = []         row in table_for_players.findall("tr"):             ldaterowindex in row.findall("th", {"colspan" : "4"}):                  if ldaterowindex.text == index:                     ldateinfomatchcase = true                 else:                     ldateinfomatchcase = false               if ldateinfomatchcase == true:                      linforowindex in row.findall("td", {"class": "info"}):                          link in linforowindex.findall("a", {"class" : "preview"}):                              lpreviewlinklist.append("http://www.afl.com.au/" + link.get('href'))         print lpreviewlinklist         gdict[index] = lpreviewlinklist

my main aim player names playing match in home , in away team according date in data structure.

i prefer using css selectors. select first table, rows in tbody ease of processing; rows 'grouped' tr th rows. there can select next siblings don't contain th headers , scan these preview links:

previews = {}  table = soup.select('table.fixture')[0] group_header in table.select('tbody tr th'):     date = group_header.string     next_sibling in group_header.parent.find_next_siblings('tr'):         if next_sibling.th:             # found next group, end scan             break         preview in next_sibling.select('a.preview'):             previews.setdefault(date, []).append(                 "http://www.afl.com.au" + preview.get('href'))

this builds dictionary of lists; current version of page produces:

{u'monday, june 09': ['http://www.afl.com.au/match-centre/2014/12/melb-v-coll'],  u'sunday, june 08': ['http://www.afl.com.au/match-centre/2014/12/gcfc-v-syd',                       'http://www.afl.com.au/match-centre/2014/12/fre-v-adel',                       'http://www.afl.com.au/match-centre/2014/12/nmfc-v-rich']}

Comments

Post a Comment

Popular posts from this blog

C# random value from dictionary and tuple -

-
i have following code, , need random item contains string "region_1" dictionary. public static dictionary<int, tuple<string, int, currencytype>> itemarray = new dictionary<int, tuple<string, int, currencytype>>() { { 0xb3e, new tuple<string, int, currencytype>("region_1", 1500, currencytype.fame) } }; public static int generateitemid(string shopid) { var generateditem = itemarray.elementat(new random().next(0, itemarray.count)).key; } how select this? it's not possible efficiently... first create static random @ class level... prevent non-random behaviour if run query on short period of time... (it's seeded discrete clock) static random rnd = new random(); then: var item = itemarray.values .where(t => t.item1 == shopid) .orderby(_ => rnd.next()) .firstordefault()
Read more

cgi - How do I interpret URLs without extension as files rather than missing directories in nginx? -

-
so i'm trying install zoneminder , have run under nginx, has few compiled files need run using fast-cgi. these files lack extension. if file has extension, there no issue , nginx interprets file , return 404 if can't find it. if has no extension, assume it's directory , return 404 when can't find sort of index page. here have now: # security camera domain. server { listen 888; server_name mydomain.com; root /srv/http/zm; # enable php support. include php.conf; location / { index index.html index.htm index.php; } # enable cgi support. location ~ ^/cgi-bin/(.+)$ { alias /srv/cgi-bin/zm/; fastcgi_pass unix:/run/fcgiwrap.sock; fastcgi_param script_filename $document_root/$1; include fastcgi.conf; } } the idea is, under cgi-bin directory goes through fastcgi pipe. any idea on howi can fix this? upon closer inspection, realized zoneminder has 2 cgi sc...
Read more

.htaccess - htaccess convert request to clean url and add slash at the end of the url -

-
ok , searched more 4 days , more 20 posts here in stach or in websites without kind of success first here htaccess file rewriteengine on rewriterule ^([a-za-z0-9_]+)/?$ index.php?page=$1 [nc] rewriterule ^([a-za-z0-9_]+)\/([a-za-z0-9_]+)\/?$ index.php?page=$1&action=$2 [nc] i tried many codes 1 worked me can call request , works me www.example.com/pagename => www.example.com/index.php?page=pagename here questions , please provide me information "in detail" understand code well 1- how convert request automatically www.example.com/index.php?page=pagename www.example.com/pagename 2- if tried add slash @ end shows me page without images or styling mean www.example.com/pagename => works www.example.com/pagename/ => doesn't work that's :) for first part need 301 rule this: rewritecond %{the_request} /index\.php\?page=([^\s&]+) [nc] rewriterule ^ /%1? [r=301,l] for second part use absolute path in css, js, images files r...
Read more