beautifulsoup - Extracting data from a web page using BS4 in Python -

-

i trying extract data site: http://www.afl.com.au/fixture

in way such have dictionary having date key , "preview" links values in list, like

dict = {saturday, june 07: ["preview url-1, "preview url-2","preview url-3","preview url-4"]}

please me it, have used code below:

def extractdata():     ldateinfomatchcase = false #     ldateinfomatchcase = []     global gdict     row in table_for_players.findall("tr"):         ldaterowindex in row.findall("th", {"colspan" : "4"}):             ldatelist.append(ldaterowindex.text)      print ldatelist     index in ldatelist:         #print index         lpreviewlinklist = []         row in table_for_players.findall("tr"):             ldaterowindex in row.findall("th", {"colspan" : "4"}):                  if ldaterowindex.text == index:                     ldateinfomatchcase = true                 else:                     ldateinfomatchcase = false               if ldateinfomatchcase == true:                      linforowindex in row.findall("td", {"class": "info"}):                          link in linforowindex.findall("a", {"class" : "preview"}):                              lpreviewlinklist.append("http://www.afl.com.au/" + link.get('href'))         print lpreviewlinklist         gdict[index] = lpreviewlinklist

my main aim player names playing match in home , in away team according date in data structure.

i prefer using css selectors. select first table, rows in tbody ease of processing; rows 'grouped' tr th rows. there can select next siblings don't contain th headers , scan these preview links:

previews = {}  table = soup.select('table.fixture')[0] group_header in table.select('tbody tr th'):     date = group_header.string     next_sibling in group_header.parent.find_next_siblings('tr'):         if next_sibling.th:             # found next group, end scan             break         preview in next_sibling.select('a.preview'):             previews.setdefault(date, []).append(                 "http://www.afl.com.au" + preview.get('href'))

this builds dictionary of lists; current version of page produces:

{u'monday, june 09': ['http://www.afl.com.au/match-centre/2014/12/melb-v-coll'],  u'sunday, june 08': ['http://www.afl.com.au/match-centre/2014/12/gcfc-v-syd',                       'http://www.afl.com.au/match-centre/2014/12/fre-v-adel',                       'http://www.afl.com.au/match-centre/2014/12/nmfc-v-rich']}

Comments

Post a Comment

Popular posts from this blog

database - VFP Grid + SQL server 2008 - grid not showing correctly -

-
i having problem, have grid called "gridbasica" , this: thisform.gridbase.recordsource = "sgviemp" sgviemp cursor made executing select sqlserver command sqlexec. when want modify registry, go form , the update there. so, when come form grid, grid isnt grid anymore, big white rectangle. reading around said: delete recordsource , asingt again. thisform.gridbase.recordsource = "" *--i sqlexe here thisform.gridbase.recordsource = "sgviemp" i , works, but, when click button modify again cursos updated modify form doesnt load fields did first time. could me out that? thanks in advance. --------------- edit ------------------ i have been doing recommended: lnselect = select(0) n = sqlexec(thisform.conexion,"select emp_ccodigo codigo, emp_cnombre 'nombre o razon social', emp_cnumrif 'r.i.f' sgviempr","sgviemp1") set echo on suspend select sgviemp zap in sgviemp *-- browse *--zaps correct
Read more

jquery - Set jPicker field to empty value -

-
im using jpicker color selector in form. have clear value set window/icon 'no color selected' state. i've tried following: $(':input[name="color"]').val(''); //access value directly. (does not affect jpicker) 2: $.jpicker.list[0].color.active.val('hex', '', this); //set color black , input value 000000, instead of empty string 3: $.jpicker.list[0].color.active.val(null, '', this); //does nothing none of 3 methods seems behave properly. in advance. this works me :) $.jpicker.list[0].color.active.val('v', null); here example: http://jsfiddle.net/3up7n/
Read more

.htaccess - htaccess convert request to clean url and add slash at the end of the url -

-
ok , searched more 4 days , more 20 posts here in stach or in websites without kind of success first here htaccess file rewriteengine on rewriterule ^([a-za-z0-9_]+)/?$ index.php?page=$1 [nc] rewriterule ^([a-za-z0-9_]+)\/([a-za-z0-9_]+)\/?$ index.php?page=$1&action=$2 [nc] i tried many codes 1 worked me can call request , works me www.example.com/pagename => www.example.com/index.php?page=pagename here questions , please provide me information "in detail" understand code well 1- how convert request automatically www.example.com/index.php?page=pagename www.example.com/pagename 2- if tried add slash @ end shows me page without images or styling mean www.example.com/pagename => works www.example.com/pagename/ => doesn't work that's :) for first part need 301 rule this: rewritecond %{the_request} /index\.php\?page=([^\s&]+) [nc] rewriterule ^ /%1? [r=301,l] for second part use absolute path in css, js, images files r
Read more