首页 诗词 字典 板报 句子 名言 友答 励志 学校 网站地图
当前位置: 首页 > 教程频道 > 开发语言 > 编程 >

Hadoop 中的Configuration种

2012-12-22 
Hadoop 中的Configuration类Configiration类:javadoc中是这么说的:Provides access to configuration para

Hadoop 中的Configuration类
Configiration类:
javadoc中是这么说的:
Provides access to configuration parameters.

Resources(xml文件,先加载default xml,而后,加载自定义的xml)

Configurations are specified by resources. A resource contains a set of name/value pairs as XML data. Each resource is named by either a String or by a Path. If named by a String, then the classpath is examined for a file with that name. If named by a Path, then the local filesystem is examined directly, without referring to the classpath.

Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath:

core-default.xml : Read-only defaults for hadoop.
core-site.xml: Site-specific configuration for a given hadoop installation.
Applications may add additional resources, which are loaded subsequent to these resources in the order they are added.
Final Parameters(如果定义属性为final则后来加载的key-value不能覆盖)

Configuration parameters may be declared final. Once a resource declares a value final, no subsequently-loaded resource can alter that value. For example, one might define a final parameter with:

  <property>
    <name>dfs.client.buffer.dir</name>
    <value>/tmp/hadoop/dfs/client</value>
    <final>true</final>
  </property>
Administrators typically define parameters as final in core-site.xml for values that user applications may not alter.
Variable Expansion(可以用${}格式,使用前面的属性作为变量)

Value strings are first processed for variable expansion. The available properties are:

Other properties defined in this Configuration; and, if a name is undefined here,
Properties in System.getProperties().
For example, if a configuration resource contains the following property definitions:

  <property>
    <name>basedir</name>
    <value>/user/${user.name}</value>
  </property>
 
  <property>
    <name>tempdir</name>
    <value>${basedir}/tmp</value>
  </property>
When conf.get("tempdir") is called, then ${basedir} will be resolved to another property in this Configuration, while ${user.name} would then ordinarily be resolved to the value of the System property with that name.

我想说,我看完我看源码的一点感悟:
1、每一个类当需要configuration类来配置自己时候,只需要extends Configured,这样我们就可以使用Configuration来配置自己了。需要单独的配置来时只需要,在使用configuration之前单独把自己的配置文件load进来就行,就像sqoop的配置文件sqoop-site.xml:

static {    Configuration.addDefaultResource("sqoop-site.xml");  }

这样就可以使用。
3、configuration类中定义了各种 get×方法,例如,我们需要某个属性的时候只需要调用其中的一个get方法,即可。

hadoop使用configuration类,设计的很灵活,能让我们通过简单的xml非常灵活配置我们参数。

热点排行