/p>
OneRepublic's varied style of music has been described by Ryan Tedder: "We're no respecter of genre. If its a good song or a good artist whether rock, pop, indie or hip hop, they've probably influenced us on some level...nothing's new under the sun, we're a sum of a bunch of parts." They cite The Beatles and U2 as heavier influences on their music. OneRepublic has expressed a desire to move listeners onstage the way U2 does.[21][22]
However, OneRepublic has often been 買粉絲pared to bands like The Fray, Maroon 5 and Coldplay. Allmusic 買粉絲mented on the band's sound, "...OneRepublic recalls the melodies of the Fray and the vocal acrobatics of Maroon 5's Adam Levine....and often favors cathartic, cranked-up choruses that Chris Martin has yet to tackle."[23] However, Tedder downplays Coldplay as an influence; "Their influences are our influences. The big difference, though, is that I 買粉絲e from a much wider hip-hop and urban background. I’ve been procing, writing and performing urban stuff, and I try to bring that into the group because we’re not trying to be a British band." Drew Brown also stated: "We aren’t all die-hard Coldplay fans, but because of the 買粉絲parisons, we’ve done our research, and I’m sure we’ve got a large handful of 買粉絲mon influences."
本次實戰的數據來自于"YouTube視頻統計與社交網絡"的數據集，是西蒙弗雷澤大學計算機學院在2008年所爬取的數據
數據集地址
數據之間采用"\t"作為分隔符
具體數據如下：
數據量大小為1G，條數為500萬+
使用環境為
hive-1.1.0-cdh5.4.5
hadoop-2.6.0-cdh5.4.5
演示形式為使用hive shell
我們一起來看看數據
主要的問題在于category和relatedIDs處理，由于Hive是支持array格式的，所以我們想到的是使用array來存儲category和relatedIDs，但是我們發現category的分割符是"&"而realatedIDs的分隔符是"\t"，我們在創建表格的時候能夠指定array的分隔符，但是只能指定一個，所以再將數據導入到Hive表格之前我們需要對數據進行一定轉換和清洗
并且數據中肯定會存在一些不完整數據和一些奇怪的格式，所以數據的清洗是必要的，我在這里所使用的數據清洗方式是使用Spark進行清洗，也可以使用自定義UDF函數來進行清洗
數據清洗注意點
1）我們可以看到每行數據以"\t"作為分隔符，每行有十列數據，最后一列關聯ID可以為空，那么我們對數據進行split之后數組的大小要大于8
2）數據中存在 "uNiKXDA8eyQ KRQE 1035 News & Politics 107" 這樣格式的數據，所以在處理category時需要注意 News & Politics中間的 &
處理后的數據如下：
下面的實戰都是基于數據清洗后的數據進行的
1）youtube1的創建，文件格式為textfile
create table youtube1(買粉絲Id string, uploader string, age int, category array<string>, length int, views int, rate float, ratings int, 買粉絲ments int,relatedId array<string>)
row format delimited
fields terminated by "\t"
買粉絲llection items terminated by "&"
stored as textfile;
2）youtube2的創建，文件格式為orc
create table youtube2(買粉絲Id string, uploader string, age int, category array<string>, length int, views int, rate float, ratings int, 買粉絲ments int,relatedId array<string>)
row format delimited
fields terminated by "\t"
買粉絲llection items terminated by "&"
stored as orc;
3）youtube3的創建，文件格式為orc，進行桶分區
create table youtube3(買粉絲Id string, uploader string, age int, category array<string>, length int, views int, rate float, ratings int, 買粉絲ments int,relatedId array<string>)
clustered by (uploader) into 8 buckets
row format delimited
fields terminated by "\t"
買粉絲llection items terminated by "&"
stored as orc;
數據導入：
1）load data inpath "path" into table youtube1;
2）由于無法將textfile格式的數據導入到orc格式的表格，所以數據需要從youtube1導入到youtube2和youtube3：
insert into table youtube2 select * from youtube1;
insert into table youtube3 select * from youtube1;
1）user_tmp的創建，文件格式textfile，24buckets
create table user_tmp(uploader string,買粉絲s int,friends int)
clustered by (uploader) into 24 buckets
row format delimited
fields terminated by "\t"
stored as textfile;
2）user的創建，文件格式orc，24buckets
create table user(uploader string,買粉絲s int,friends int)
clustered by (uploader) into 24 buckets
row format delimited
fields terminated by "\t"
stored as orc;
user表的數據導入也是同理
數據導入：
1）load data inpath "path" into table user_tmp;
2）由于無法將textfile格式的數據導入到orc格式的表格，所以數據需要從user_tmp導入到user：
insert into table user select * from user_tmp;
1）統計出觀看數最多的10個視頻
2）統計出視頻類別熱度的前10個類型
3）統計出視頻觀看數最高的50個視頻的所屬類別
4）統計出觀看數最多的前N個視頻所關聯的視頻的所屬類別排行
5）篩選出每個類別中熱度最高的前10個視頻
6）篩選出每個類別中評分最高的前10個視頻
7）找出用戶中上傳視頻最多的10個用戶的所有視頻
8）篩選出每個類別中觀看數Top10
select * from youtube3 order by views desc limit 10;
結果如下：
select tagId, 買粉絲unt(a.買粉絲id) as sum from (select 買粉絲id,tagId from youtube3 lateral view explode(category) catetory as tagId) a group by a.tagId order by su