BMTD 's Yard of Fun

Technology, Sports, Music, Chinese Essays

Browsing Posts tagged java



Since this problem seems to popup on different lists, this message has
been cross-posted to the general Red Hat discussion list, the RHEL3
(Taroon) list and the RHEL4 (Nahant) list. My apologies for not having
the time to post this summary sooner.

I would still be banging my head against this problem were it not for
the generous assistance of Tom Sightler <ttsig@xxxxxxxxxxxxx> and Brian
Long <brilong@xxxxxxxxx>.

In general, the out of memory killer (oom-killer) begins killing
processes, even on servers with large amounts (6Gb+) of RAM. In many
cases people report plenty of “free” RAM and are perplexed as to why the
oom-killer is whacking processes. Indications that this has happened
appear in /var/log/messages:
Out of Memory: Killed process [PID] [process name].

In my case I was upgrading various VMware servers from RHEL3 / VMware
GSX to RHEL4 / VMware Server. One of the virtual machines on a server
with 16Gb of RAM kept getting whacked by the oom-killer. Needless to
say, this was quite frustrating.

As it turns out, the problem was low memory exhaustion. Quoting Tom:
“The kernel uses low memory to track allocations of all memory thus a
system with 16GB of memory will use significantly more low memory than a
system with 4GB, perhaps as much as 4 times. This extra pressure
happens from the moment you turn the system on before you do anything at
all because the kernel structures have to be sized for the potential of
tracking allocations in four times as much memory.”

You can check the status of low & high memory a couple of ways:

# egrep 'High|Low' /proc/meminfo
HighTotal:     5111780 kB
HighFree:         1172 kB
LowTotal:       795688 kB
LowFree:         16788 kB

# free -lm
total       used       free     shared    buffers     cached
Mem:          5769       5751         17          0          8       5267
Low:           777        760         16          0          0          0
High:         4991       4990          1          0          0          0
-/+ buffers/cache:        475       5293
Swap:         4773          0       4773

When low memory is exhausted, it doesn’t matter how much high memory is
available, the oom-killer will begin whacking processes to keep the
server alive.

There are a couple of solutions to this problem:
continue reading…

每次运行之前需要在数据库里populate需要的测试数据,这个可以用SQL ant task自动在test case里实现--相关test case第一步就是作数据初始化。

步骤:

(1) 建议先写好SQL script,把这些sql文件放在单独的SQL目录里。(optional)

(2) 把 mysql jdbc driver 文件 “mysql-connector-java-5.0.4-bin.jar”拷贝到 webtest的lib目录下,并且加入到java classpath (或者 webtest.sh /webtest.bat 的calsspath里)

哪里找文件 mysql-connector-java-5.0.4-bin.jar呢?

--先从这里下载mysql-connector-java-5.0.4.zip : http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.0.4.zip/from/http://mirror.services.wisc.edu/mysql/ ,解压后你会看到mysql-connector-java-5.0.4-bin.jar.

(3)写SQL task:

一个简单的SQL ant task是这样的:

<sql
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://host:port/database"
userid="sa"
password="pass"
src="data.sql"
/>

意思是链接到位于 host:port的mysql数据库里的"database" schema, 用sa/pass登陆,然后运行 "data.sql"文件。

或者这样直接写sql语句:

<sql
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://host:port/database"
userid="sa"
password="pass"
><![CDATA[

update some_table set column1 = column1 + 1 where column2 < 42;

]]></sql>

或者这样运行几个sql文件:

<sql
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://host:port/database"
userid="sa"
password="pass" >
<transaction src="data1.sql"/>
<transaction src="data2.sql"/>
<transaction src="data3.sql"/>
<transaction>
truncate table some_other_table;
</transaction>
</sql>

(4) 在你的webtest里把这个task加进来 :

<target name="testit">
<!–初始化数据 -->
<sql
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://host:port/database"
userid="sa"
password="pass" >
<transaction src="data1.sql"/>
<transaction src="data2.sql"/>
<transaction src="data3.sql"/>
<transaction>
truncate table some_other_table;
</transaction>
</sql>

<webtest name="myTest">
<config host="localhost" port="8080" protocol="http" basepath="" summary="true" saveresponse="true" />

<steps>
<invoke description="a" url="sb.php" />
<!–<verifyCookie name="AuthJ" text="four498@gmail.com~wangluo~1161665636" /> –>
<!–<invoke description="a" url="sb.php?a=1" /> –>

</steps>
</webtest>
</target>

上面只是简单例子,你们做时这个<sql>部分可以放到一个include文件里, 在test case里 include进来。

这是我们实现Agile development重要的一环.

(0) 安装

— download jave runtime (JRE)
— download webtest: http://webtest.canoo.com/webtest/build.zip, 解压到本地目录 <WEBTEST_HOME> (例如, C:\webtest)
— Add <WEBTEST_HOME>\bin to your PATH (控制面板–>系统–>高级–>环境变量)
–cd <WEB_HOME>\doc\samples 运行:
webtest -buildfile installTest.xml

一般来说,运行test cases用这样的语法:
webtest -buildfile <testcasefile>

例如,

webtest -buildfile mytest.xml

(1)test cases目录结构

建立一个目录叫testcases,testcases放到该目录下面

testcases
includes: 包含文件
properties: 存储一些与环境相关或常变化的测试数据
modules:小的可重用的测试模块
UseCases : 对应于use cases的test cases
TestResults: 存放测试结果和报告.

建立一个 web directory 指向TestResults目录. 例如, http://localhost/TestResults/testresult.html 就可以看见测试报告.

(2) 简单例子:

附件这个简单test case是验证海归网登陆的:

把这个文件存为mytest1.xml, (去掉.doc后缀), 放到TestCases目录下,把name="webtest.home" location="E:/canoo" 中location的值改为你的canoo web test的根目录.

然后运行 webtest -buildfile mytest1.xml;运行完毕后到http://localhost/TestResults/results.html看测试报告.

最后一步应该是失败的,因为crazytang的password不对. 要想测试测试成功需要把上面login.ok.password的值改成crazytang的密码.

注:测试报告格式和语言可以通过XSLT定制;回头我们会修改一下现在的报告格式,并且加上email通知功能.

(3) Canoon WebTest 的功能简介

背景: canoo webtest是用java 编写的,我们上述的例子用 ant 执行.(ant本身是一个java build tool, 相当于make,但比make强的多)

它是一个web acceptance工具,而不是一个junit/htmlunit/phpunit之类的unit test工具,也就是说进行黑箱测试. 在用于开发时.需要预先根据
详细需求编写test cases(实际上也是use cases)把期望的web site什么样子有什么功能表达出来.

test cases是xml格式的,一般来说需要手工编写 (如果是对现有网站测试,可以用工具自动把用户手工测试的过程记录下来自动生成test cases,但对于新项目开发来说没有什么用处);

(4)test case编写入门

看看上面的简单例子.
每个<webtest> 元素代表一个 web session,里面包括<config>和<steps>. 前者定义访问的网站属性,后者是一系列测试的步骤,例如采取什么行动(点击button,设置cookie等),或者验证
返回页面(标题,内容,图象, response code等等);每个步骤都有一个相对应的webtest step. webtest当前所有支持的步骤主要见见:

http://webtest.canoo.com/webtest/manual/syntaxCore.html
http://webtest.canoo.com/webtest/manual/syntaxExtension.html

此外还有关于email/pdf/excel等的验证,config等的语法,详见手册: http://webtest.canoo.com/webtest/manual/manualOverview.html

(5) property, 包含文件等:

可以定义类似变量的property, 例如我上面简单例子中引用的${user}, ${login.ok.password}等都是在文件开始定义的properties.详细的propeties用法
见http://webtest.canoo.com/webtest/manual/properties.html

test cases可以由许多可重用的模块组成,这样易于维护减少冗余. 详细请见: http://webtest.canoo.com/webtest/manual/samples.html.

(6) 如果有必要直接在数据库中准备测试数据,(例如测试前需要建立一批用户)可以用与SQLUNIT结合实现:

详见 http://webtest.canoo.com/webtest/manual/sqlunit.html

(7)这里有些用户贡献的工具或测试步骤: http://webtest-community.canoo.com/wiki/space/Contributions+and+Uploads

(8) 我们会设置一个cron job(scheduled task)每夜自动运行所有test cases.

(9) test cases也存入SVN中.

Visibroker Naming Service (VBNS) is the Corba naming services provided by Borland Visibroker. It supports implicit object clustering which allows you to bind multiple object references under one name. This is not CORBA compliant, how ever it allows easy/transparent clustering of corba objects instead of explictly going through the hassle of ClusterManager.

Several things to note for implicit object cluster mode:
(1) if the peroperty “vbroker.naming.propBindOn” is set to 1, the implicit clustering feature is turned on. by default it is 0.

(2) if the peroperty “vbroker.naming.smrr.pruneStaleRef” is set to 1, a stale object reference that was previously bound to a cluster with the Smart Round Robin criterion will be removed from the bindings when the name service discovers it.

The “prineStaleRef” behavior sometimes causes trouble though: The Naming Service in implicit clustering mode might mark an active object reference as stale erroneously due to some transient network problem that caused the object to be unreachable for some time.

For example, the following scenario happened:
— naming service and the corba server are running on the same box,
— between the Corba server box and the corba client there is a firewall
— the corba client is able to looks up the object A from VBNS and successfully got a references
— the corba client tries to connect to the the Server, however was blocked by the firewall and can’t talk to it. so it reports back to the naming service with a “MarkSuspect” message marking the object as stale.
— the naming service then considers the object refernce A as stale and remove (unbind) it from the cluster.
–now other corba clients can no longer get a refrence of object A although the Server is still actively running.

The problem above is because of hte firewall, which should not block the traffic between the corba client and the corba server. but it’s not easy to identify the root cause. Similar problem is also possible to occured becasue of transient network issues. The article at http://support.borland.com/entry.jspa?externalID=3355&categoryID=124 presnts an out-of-band solution for this but it does not address the problem well since the code will bind multiple refrences of the same object in the cluster.

In Spring IOC framework you can define bean properties of some collection types, such as List, Set, Propertie and Map. For example, for a bean that has a property “theList” of type “java.util.ArrayList”, you can specify the value as below:

&lt;bean id="exampleBean" class="examples.ExampleBean"&gt;
&nbsp;&nbsp;  &lt;property name="theList"&gt;
&nbsp; &nbsp; &nbsp; &nbsp;       &lt;list&gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;          &lt;value type="java.lang.String"&gt;a String&lt;/value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;          &lt;value type="java.lang.Interger"&gt;100&lt;/value&gt;
&nbsp; &nbsp; &nbsp; &nbsp;       &lt;/list&gt;
&nbsp; &nbsp;    &lt;/property&gt;
&lt;/bean&gt;

This is fine. But if the “theList” is a java.util.Vector then the above won’t work because by default the List FactoryBean uses an ArrayList and can’t convert a java.util.ArrayList to java.util.Vector– you will get a conversion exception.

It seems Spring does not support “Vector” property type directly. You have to either define a custom PropertyEditor for Vector, or, work around it with something like below:

&lt;property name="theList" &gt;
&nbsp; &nbsp;              &lt;bean class="java.util.Vector"&gt;
&nbsp; &nbsp; &nbsp; &nbsp;                     &lt;constructor-arg &gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;                          &lt;value type="int"&gt;2&lt;/value&gt;
&nbsp; &nbsp; &nbsp; &nbsp;                     &lt;/constructor-arg&gt;
&nbsp; &nbsp;               &lt;/bean&gt;
  &lt;/property&gt;
  &lt;property name="theList[0]" &gt;
&nbsp; &nbsp;             &lt;value type="java.lang.String"&gt;a String&lt;/value&gt;
  &lt;/property&gt;
  &lt;property name="theList[1]" &gt;
&nbsp; &nbsp;             &lt;value type="java.lang.Integer"&gt;100&lt;/value&gt;
  &lt;/property&gt;

Or a better way like this:

&lt;property name="theList" &gt;
&nbsp; &nbsp;              &lt;bean class="java.util.Vector"&gt;
&nbsp; &nbsp; &nbsp; &nbsp;                     &lt;constructor-arg &gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;                          &lt;list&gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;                             &lt;value type="java.lang.String"&gt; a String&lt;/value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;                             &lt;value type="java.lang.Integer"&gt; 100  &lt;/value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;                          &lt;/list&gt;
&nbsp; &nbsp; &nbsp; &nbsp;                     &lt;/constructor-arg&gt;
&nbsp; &nbsp;               &lt;/bean&gt;
  &lt;/property&gt;