By
R. W. M. Johnson (New Zealand)
In the last decade public agricultural research and extension have been reorganised in New Zealand. The reorganisation has been driven by fiscal austerity and efficiency goals as part of a wider reform of government services. In this reform process there has been a concentration on greater accountability, financial probity, principal/agent relationships, divided responsibility, and privatisation (in some cases). In the agricultural science sector the former state departments have been replaced by a new structure dividing the functions of funders, purchasers and providers into separate entities, while extension services have been transferred successively from a state department (Agriculture) to a stand-alone bureau, and then sold to a private company. This paper reviews and analyses the previous government structures that were found to be unacceptable for both agricultural science and extension, the structures that were put in their place, and the organisational models that were explicitly or implicitly employed in deriving the new structures.
Key words: agricultural science, extension, reform, organisational models
R.W.M.Johnson C/- MAF, Policy PO Box 2526, Wellington, New Zealand. Ph:<PHONE_NUMBER> Fax:<PHONE_NUMBER>, E-mail<EMAIL_ADDRESS>Contributed Paper submitted to the XXIII Conference of the International Association of Agricultural Economists, Sacramento, California, August, 1997.
Organisational Models for Management of Research and Extension: the Case of New Zealand (science.wps)
The organisation of government services in New Zealand were previously based on the British Westminster model with a centralised Parliament and cabinet system supplied by a large bureaucracy. From the 1930s the degree of government intervention in economic and social affairs was at an increasing level; necessitated in New Zealand's case by a dependence on agricultural exports and imports of manufactures. Economic institutions were protected from international fluctuations by tariffs and import controls and internal marketing schemes. At the same time a comprehensive welfare net was created based on tax-payer funds. Economic management was centralised and government services reached into the lives of most people.
Within this historic framework, the provision of science and extension services was also part of the accepted system. A Department of Agriculture was established in 1892 to manage and report on emerging problems of disease control, pests and quality assurance for the newly discovered refrigerated meat and butter trade. Over the years the Department expanded to include agricultural experimentation, export inspection systems, farm training and extension services. In 1926 a Department of Scientific and Industrial Research (DSIR) was established to organise science across all government departments (including some agricultural science functions previously managed by the Department of Agriculture-now Ministry of Agriculture (MAF)). These two departments were state funded, scientists were state employees, and the structure of command was centralised and autocratic. As part of the DSIR mandate, however, an additional funding provision was made to encourage industry research associations on a 50:50 support basis shared with the industry concerned. In this way relatively large post-harvest research organisations were established for dairy products, wool, leather and meat, and an input supply research organisation for fertiliser research.
Reforms in the science sector as a whole began in 1985 with instructions to the two departments to introduce charges for science and extension services and to achieve 2% savings per year in government funding for the next five years. The aim was to encourage alternative funding ('commercial revenue targets') (MRST 1992a). In 1989, it was decided to create a Ministry of Research, Science and Technology (MRST) to take over science policy functions, and a Foundation for Research, Science and Technology (FRST) to select and fund research projects which deserved public support (science purchasers). The operation of science remained in the old departments temporarily but the existing administrative funding was divided between the new policy Ministry and the Foundation. Finally on 1 July 1992, it was decided to reform the operational or provider side of science as well, and sector research institutes were established to take over these functions from the former departments (science providers). Five agricultural research institutes were created for the pastoral, crop, horticultural, landcare and water subject areas. The FRST was to administer a public good science fund (PGSF) with specific criteria for identifying projects worth funding in the national interest.
The system of science organisation was characterised by public control, departmental bureaucracy, large scale, multi purpose, vertically integrated, administratively pluralistic, and long chains of command (Boston et al 1996) as far the two major departments were concerned. The research associations were, by contrast, self-contained, specialised and self governing, and partially privately funded.
There was a degree of duplication between the two largest organisations involved, the staff of the two organisations were dominated by the public service ethic of working hours, holidays, and retirement provisions, and the results of their efforts were not sufficiently publicly visible. Departments were funded on a cash flow and annual appropriation basis with weak reporting systems to central government (Upton 1995). Accounting systems emphasised the cost of inputs (wages, salaries and overheads), and not the cost of results or outputs. Objectives were blurred between private good and public good science and success was often measured on the appropriable private good side. Operational research (support for a department's objectives) was not separately funded or identified from proper public good research [fundamental or basic research] and policy advice functions were delivered to government from within the bureaucracy that provided the science and extension services (provider capture).
Another perception was that the departments and agencies made their own priority-setting decisions independently of each other (MRST 1992a, p.9). There was no common framework for reporting science activity and expenditure. A common framework was only introduced in 1990-91 when all the individual science budgets were amalgamated. Within departments and agencies peer review systems and publication incentives were said to work well.
From a public choice viewpoint, the science departments, in common with other departments, were considered to be beset by self-interest philosophies particularly with regard to working conditions and lack of public accountability. They also suffered, in common with others, from long control hierarchies, and specific to science and extension, dispersed units of service provision. The long hierarchy generated high overhead costs to all science provider activities which technically made the science service more expensive than it need be. Finally, the public service system of financial management (based on cash flows alone) lead to poor understanding of relevant costs and benefits of different activities.
Public service reform was part of a general plan to reduce the involvement of government in commerce, and to introduce market driven methods for the provision of government services (Schwartz 1996, p.l). Some departments with large trading enterprises lent themselves to privatisation. Other departments were subject to separation of policy and delivery services, while others were given less resources to achieve the objectives laid down by Ministers. All departments were subject to financial reform (Public Finance Act 1989) which mimic private enterprise disciplines through accrual accounting methods and demands for positive rates of return on capital. All departments were also put on arms-length terms with their Ministers by the imposition of formal contracts between Minister and CEO, and between CEO and department (State Sector Act 1988).
From a public choice point of view, the reforms can be seen as endorsing a conservative political agenda (minimising the role of the state, selling the state's commercial assets, curbing the functions of government agencies, and maximising liberty-understood in the limited sense of freedom from state coercion) (Boston et al 1996, p.18). Further, because politicians can be expected to abuse their power, it is argued that they should be prevented, if necessary by constitutional changes, from running budget deficits or imposing taxes beyond a certain level (Brennan and Buchanan 1985). Similarly, because departments have a vested interest in their own survival, they should not tender advice and implement policy (Treasury 1987, p.75). Otherwise, their advice will be biased and bureaucratic capture might occur. Finally all services provided by the state should be made as contestable as possible (Boston et al 1996, p.18).
The problem of improving performance in a public service is central to the reforms. The route taken was to introduce contractual obligations between Minister and CEO that could be monitored and indeed enforced on occasion. CEOs were employed on limited period contracts subject to review. This can be seen as an application of agency theory where the interests of principal and agent are bound to conflict. Although originally applied to problems arising from the separation of ownership and control in firms, it has been much more widely applied. Boston quotes Jensen and Meckling (1976, p. 309) as follows:
The problem of inducing an agent to behave as if he [or she] were maximising the principal's welfare is quite general. It exists in all organisations and in all cooperative efforts, at every level of management in firms, in universities, in mutual companies, in co-operatives, in governmental authorities and bureaus, and in relationships normally classified as agency relationships such as are common in the performing arts and the market for real estate.
In the political realm voters can be regarded as principals who contract with politicians (their agents) to undertake a range of tasks and activities on their behalf (Boston et al, p.19, Moe 1984, p.765):
... the whole of politics can be seen as a
chain of principal-agent relationships, from citizen to politician
to bureaucratic superior to bureaucratic subordinate and on down
the hierarchy of government to the lowest level bureaucrats who
actually deliver services directly to citizens. Aside from the
ultimate principal and the ultimate agent, each actor in the
hierarchy occupies a dual role in which he [or she] serves both as
principal and agent.
Now agency theory explains why organisations have difficulty in meeting their obligations in the real world. Where agents are prone to shirking, deception, cheating and collusion, obligations are not met and efficiency is reduced. In bureaucracies, organisations are slow to respond to political change. As is well known, adverse selection describes the case where the principal does not have sufficient information about an agent before agreement is reached. Moral hazard is the case where the agent's behaviour after agreement cannot be monitored or anticipated. These deficiencies lead to incomplete information in a relationship, asymmetrical information and greater uncertainty. The application to bureaucracy lies in better specification of what is to be achieved, output-based reward systems, more direct incentives and better monitoring systems.(Boston et al 1996, p.19).
A related problem concerns the comparative costs of planning, adapting and monitoring task completion under alternative governance structures (Williamson 1985, p.2). Rational agents should select governance structures that minimise their aggregate production and transaction costs. The application of this transaction cost theory is to the practice of contracting out particular services. External contracting is best when behavioural uncertainty is low, and the risks of adverse selection are minimised; when the quantity of the desired goods and services can be easily measured and thus monitored; and when the number of potential suppliers is large (Boston et al 1996, p. 24).
An organisational theory that pertains more to science organisations is the administrative doctrine concerning inclusive versus divided responsibility. Inclusive or ‘single roof organisations’ favour horizontal and/or vertical integration of activities so as to facilitate unified political responsibility, better policy coordination, and lower transaction costs. Divided responsibility organisations minimise conflicts of interest, minimise bureaucratic capture, reduce the concentration of power, enhance the clarity of the organisational mission and facilitate contestable provision (Boston et al 1996, p.75).
Managerialism (Hood 1990, Aucoin 1990) embraces the ideas that the public sector can be managed like the private sector; that results are more important than the process; that the disaggregation of large bureaucratic structures into quasi-autonomous agencies improves accountability and efficiency; that public services should be contracted out; that tightly specified contracts and time scales produce better results; and that the use of private sector management practices such as strategic plans, performance agreements, mission statements, performance-linked remuneration systems, and corporate management information systems are preferable.
The objective of reform was increased accountability, fiscal savings, and better outcomes for society. This was to be achieved by privatisation (extension), or by organisational forms that avoided bureaucratic capture and set tighter goals for performance (agricultural science). Performance could be enhanced by contestability for public science contracts, tighter organisational structures, tighter remuneration systems, and removal of the vast load of bureaucratic overheads including layers of administrators. As state-owned enterprises, research institutes became subject to private sector financial disciplines including returning a dividend to their owners. Needless to say, these changes involved a considerable degree of hardship and adjustment particularly in the number of total jobs available in the short term. Fiscal savings were also seen as an incentive to gain greater private investment in research. This objective was aided by stricter monitoring of public good spending through FRST criteria for selection of public good projects and programmes.
One observer has stated that science represented perhaps the strictest application of the functional model in all the reforms (Boston et al 1996, p.83). There is a formal split between the roles of funder, purchaser, and provider; MRST is essentially a single purpose policy ministry; FRST purchases scientific research via a competitive bidding process from a series of CRIs, tertiary institutions, and private providers and monitors the performance of providers; the monitoring of the Crown's ownership interest in the CRrs is carried out by the Crown Company Monitoring Advisory Unit (CCMAU); and only FRST has a dual policy advice [limited], purchasing and monitoring role. Strictly speaking, FRST allocates a fixed annual fund, but operational research within remaining departments is often contracted out by them in a purchase framework also (King 1996).
Objectives. There is agreement that the new system has
provided better focus for the whole science system
(Upton 1995,
Devine 1995).
The science reforms have encouraged a cultural change among
scientists and end- users. The highly mission-oriented business
culture of the 1990s, coupled with growing end-user focus of the
research community, have given research a significant role in
underpinning international competitiveness
(Devine 1995).
The public good emphasis has given a better understanding of
where the public interest lies. Clear, well thought out policy
guide-lines are needed ... relevance only has meaning within the
ideas of public returns to New Zealand ... higher priority should
be given to programmes which, as presented, and at envisaged
funding levels, are expected to give the best marginal return
measured in appropriate social, environmental or economic value
systems
(Devine 1995).
Funding: By 1994, some significant shifts in the distribution of research expenditure had occurred. Compared with the position in 1986, private sector expenditure barely increased in nominal terms and public expenditure only increased by 22% in 7 years (Table I). Public funding for agricultural research increased by 27%. Data for universities is not comparable (see note to table).
| Table 1: Estimated Trends in Total Science
Expenditure by Major Sectors $NZm |
||||
|---|---|---|---|---|
| 1986-87 | % | 1993-94 | % | |
| Private Industry | 230 | 40 | 248 | 30 |
| Universities | 70 | 12 | 234* | 28 |
| Pub Funding:Agriculture | 100 | 17 | 127 | 16 |
| Other | 180 | 31 | 216 | 26 |
| Total | 580 | 100 | 825 | 100 |
| % of GDP | 1.09 | 1.02 | ||
| * figure enlarged by a change in
methodology compared with 1986. Source: MRST 1996; NZ Yearbook 1989. |
Crowding Out: The public/private split varies widely across output classes (Table 2). Private funding is dominant in primary product processing [the research associations], materials and engineering, and infrastructural areas. The Universities dominate social science, health and fundamental research; and the public sector dominates primary production and the environment. These allocations reflect the teaching and training role of the universities and the particular output classes where public funding is needed because appropriability is lower.
| Table 2: Science Expenditure 1993-94 by Major Output Classes and Major Providers (%) | ||||
|---|---|---|---|---|
| Private | University | Public | Total | |
| Agriculture, forestry, fisheries | 12 | 12 | 76 | 100 |
| Primary products, processing | 68 | 6 | 26 | 100 |
| Materials, engineering | 54 | 16 | 29 | 100 |
| Construction, transport, energy & information services | 66 | 19 | 14 | 100 |
| Social sciences | 4 | 65 | 30 | 100 |
| Environment and resources | 5 | 17 | 78 | 100 |
| Fundamental res., Health, Defence | 9 | 78 | 17 | 100 |
| Total | 30 | 28 | 42 | 100 |
| Source: MRST 1996, p.4l. |
The main implication of the tables is that overall private sector funding had not increased to the extent that the reforms desired in the period covered. There appears to be no greater involvement of business in research funding and rather less support for theories of' crowding out' by excessive public involvement (MRST 1992b, p.15). In terms of GDP growth, total funding levels had also fallen behind, and in real terms there was an absolute decline in resources made available. These trends appear to confirm the long term decline in support for science activities noted in one public report (MRST 1992a, p.9).
There are difficulties in getting greater private involvement.
One agricultural research institute reports: While revenue [in
1995] remained almost static, it increased slightly from new and
expanded relationships with commercial interests .... non-PGSF
sources represent the greatest long term opportunities for the
growth of AgResearch. Revenue from commercial sources is expected
to increase significantly in the years ahead, as commercial
organisations, seek new agri-technologies and bio-technologies to
exploit the opportunities offered under GATT and other liberal
trade agreements
(AgResearch Annual Report 1995). In looking to
the future, Devine (1995) observes the ultimate test is whether
end-users and government become sufficiently convinced of the value
of research benefits, [that] they increase their level of
commitment
.
Down-sizing: Since 1992, there have been widespread redundancies in some specialised areas in the CRIs. In 1995- 96, it is estimated that staff losses were 4% of total staff numbers but other recruitment was taking place. This has serious implications for the loss of experienced staff and the corporate memory (King 1996). The PGSF has only been increased by small nominal amounts since 1992 hence this trend is likely to continue.
Efficiency: The pastoral CRI reported modest profits in 1995, but noted that its investment of $48m in freehold land and buildings was substantially greater than is appropriate for a technology based organisation (AgResearch 1995). Most CRIs had the advantage of shedding layers of head office staff and offices located in urban real estate areas. Some could making savings from the removal of low quality and ill-directed projects which did not suit their new missions (Devine). Savings were not achieved in the transfer of priority setting from a departmental basis to a funder/purchaser/provider basis. The transaction costs of these activities have actually increased particularly with regard to providers' (scientists) time and energy (Upton 1995, p.4. )(King 1996).
Some reform would be desirable in the way bidding costs are shared (Pearce 1995). He suggests a more equitable sharing between purchaser and provider. Reconsideration of the process could reduce transaction costs. 'An integrated package of process changes is needed to move the science purchasing and delivery system from what can be thought of as a quality control mode to a quality management mode'.
Autonomy: In anticipation of transition problems, and in recognition of the need to give CRIs some autonomy in funding, Government arranged for some of the previous public funding [about 10%] to be allocated to the CRIs outside the PGSF system (CRI report, ch. 22). This is called non-specific output funding (NSOF). This would allow a CRI to work independently of the priorities set by FRST and could include preparatory research to establish the viability of an idea, support for research teams without public funding provision, and fellowships and the like.
This can be seen as an exception to the purchaser/provider model and recognises the uncertain nature of the scientific process and mind (Upton 1995, p.4.). 'In the absence of large alternative research funders ... [like philanthropic trusts] it seemed to me prudent to leave some funding in the hands of research providers as an insurance policy against purchaser/government failure'. Another special fund was established to provide research opportunities to extremely capable scientists whose retention was in the national interest but whose interests did not fit in with any established funding category [the Marsden Fund]. As the Minister states 'it is a response to the particular nature of the process of scientific enquiry rather than an application of institutional theory'! The universities particularly welcomed this development (McCutcheon 1995, p.19).
Universities: The universities are an important part of the science sector (Table 1). They dominate basic research and social science research expenditure (Table 2). Initially they were left out of the PGSF discussions because of disagreement over the share of funds they would have to give up out of V ote:Education, their main source of funding. University research is funded implicitly by a component of the Vote: Education block grant, by grants and contracts obtained from government departments, funding agencies and industry, by commercial activities and by scholarships awarded to research students (CRI report, ch. 28).
A small contribution to the PGSF was agreed in 1993, and this opened access to public good funding for university research. Nevertheless, one writer says that this only represents 10% of research funding in his university (McCutcheon 1995, p.18). There are problems for universities in this kind of organisation of research. If research is undertaken for the primary purpose of completing a degree, it is seen as teaching-related research which should be funded out of Vote: Education; if the primary purpose of the research is to produce a public good science output then it is classified as public good research or non-teaching research with consequent access to PGSF (McCutcheon, p.18). The latter is an obvious source of support for PhD candidates if they can survive the bidding process! Since public good funding covers professional time, university staff awarded contracts find themselves in conflict with their employers who think that they are paid to teach. If the best researchers concentrate on research, the burden of teaching falls on less qualified staff and so on. Further, some support for research in universities is on a direct cost basis, with the University covering the overheads; the PGSF system undermines these donors. McCutcheon believes that user pays system is inimical to true university research objectives based on intellectual curiosity, training and disciplinary research (ibid, p.19).
Aricultural Research and Extension: The extension activity was always the responsibility of the Department of Agriculture. There was no sharing with the universities as in other countries. In 1987, the Ministry (as it then was) amalgamated its research and extension bureaus, both to save costs and overheads and to seek possible synergies (Ritchie 1995, p.3). Then in 1991, the two were separated again: research services to be split among the CRIs, and extension services to be formed into a publicly-owned trading enterprise. In 1995, this enterprise was sold to a private firm. On the 4-year synergy experiment, Ritchie says
"strong regional autonomy helped establish local initiatives early in the commercialisation process, but at the time prevented the development of a national business ethic ... the culture of a research organisation is different from a truly commercial organisation ... research culture is driven by innovation and product development while the consultancy culture is market driven and one of client service" (ibid, p.3).
On the commercialisation issue, the government took the view (under Treasury advice) that extension services were not public goods at all, and government has no need to support such a service. This is a debatable issue especially where the public good content of information services, field demonstrations and policy servicing are concerned, As Ritchie reports, his commercial organisation has had to down-size and concentrate its activities on one-to-one extension advice, sub-contracting to the science providers, training services and advice to agribusiness (Ritchie, p.5). Sub-contracting to the science providers involves field experimentation and technology transfer. This kind of interaction has considerable potential (King). The new divided responsibilities for technology transfer have left a gap compared with the old public service system. New ideas and organisational skills are needed in this area (King 1996, Ritchie 1995). The status of technology transfer could be enhanced by identifying it as a separate outcome in the PGSF, instead of disguising it as part of an individual output (King).
1. AgResearch (New Zealand Pastoral Research Institute Limited)( 1995), Annual Report 1995, Hamilton, NZ.
2. Aucion, P. (1990), Administrative Reform in Public Management: Paradigms, Principles, Paradoxes and Pendulums, Governance 3,115-137.
3. Boston, 1., Martin, J., Pallot, 1., and Walsh, P. (1996), Public Management: the New Zealand Model, Oxford University Press, Auckland.
4. Brennan, G. and Buchanan, 1. (1985), The Reason of Rules: Constitutional Political Economy, Cambridge University Press.
5. Devine, S. (1995), Purchasing Scientific Research Outputs within the New Zealand Science Structures, Public Sector 18(4),6-9.
6. Hood, C. (1990), Beyond the Public Bureaucracy State? Public Administration in the 1990s, Inaugural Lecture, London School of Economics.
7. Jensen, M., and Meckling, W. (1976), Theory of the Firm: Managerial Behaviour, Agency Costs and Ownership Structure, Journal of Financial Economics 3, 305-360.
8. King, G. (1996), Role of the Ministry of Agriculture in Science Policy, in Policy Issues and Current Status of Agricultural Sector, Post Election brief, Ministry of Agriculture, Wellington.
9. Ministerial Science Task Group (1991), Crown Research Institutes, Wellington, New Zealand.
10. Ministry of Research, Science and Technology (MRST)(1992a), Long Term Priorities for the Public Good Science Fund: a Public Discussion Paper, Wellington, New Zealand.
11. Ministry of Research, Science and Technology (1992b), Long Term Priorities for the Public Good Science Fund: Final report, Wellington, New Zealand.
12. Ministry of Research, Science and Technology (1996), New Zealand Research and Experimental Development Statistics: All Sectors 1993/94, Wellington, New Zealand.
13. Moe, T. (1984), The New Economics of Organisation, American Journal of Political Science 28, 739-775.
14. McCutcheon, S. (1995), Purchasing Research: The University Experience, Public Sector 18(4), 16-19.
15. Department of Statistics (1989), New Zealand Year Book, Wellington, New Zealand.
16. Department of Statistics (1993), New Zealand Year Book, Wellington, New Zealand.
17. Pearce, A. (1995). Contracting in the Science Sector: a Research Provider's View, Public Sector 18(4), 10-15.
18. Ritchie, I. (1995), From the Public to the Private Sector-the Agriculture New Zealand Story, Journal of the Australian and New Zealand 1nstitute of Agricultural Science 8(4), 29-31.
19. Schwartz, H.M. (1996), Public Choice Theory and Public Choices: Bureaucrats and State Reorganisation in Australia, Denmark, New Zealand and Sweden in the 1980s, http:/darwin.clas.virginia.edu-hms2f/pubchoic.
20. Treasury (1987), Government Management, Volumes 1 and II, Government Printer, Wellington.
21. Upton, S. (1995), Contracting in the Science Sector: An Overview, Public Sector 18(4), 2-5.
22. Williamson, O. (1985), The Economic 1nstitutions of Capitalism: Firms, Markets, Relational Contracting, Free Press, New York.


LongLongIntTypeSymbol Class
Definition
Line
224
of
cppsymbols/SimpleTypeSymbol.cm
Inheritance
Constructors & Destructor
public
nothrow
LongLongIntTypeSymbol
(
const
Span
& span_)
Member Functions
public
override
void
Accept
(
SymbolVisitor
& visitor)
public
override
nothrow
String
<
uchar
>
FullName
()
const
Constructor& Destructor Details
LongLongIntTypeSymbol Constructor
public
nothrow
LongLongIntTypeSymbol(
const
Span
& span_)
Definition
Line
226
of
cppsymbols/SimpleTypeSymbol.cm
Member Function Details
Accept Member Function
public
override
void
Accept(
SymbolVisitor
& visitor)
Definition
Line
229
of
cppsymbols/SimpleTypeSymbol.cm
FullName Member Function
public
override
nothrow
String
<
uchar
> FullName()
Definition
Line
233
of
cppsymbols/SimpleTypeSymbol.cm


I have been programming for over 5 years, having started by programming small mobile apps when I was 15 years old. Realising the extent of my capabilties, I decided to publish my first mobile app, Golf Pads. It came to be my spotlight portfolio piece at the time, securing me an unconditional place at Falmouth University. At Falmouth, I studied BSc(Hons) Computing for Games, where my passion for game development continued to blossom. I gained the experience of working in a multidisciplinary team within a studio environment and explored many fields and areas within computer science, such as artificial intelligence, distributed systems, cyber security - the list goes on. Throughout my time at Falmouth, I took the role of Lead Programmer in all of my group development projects and games, to exercise my leadership and management skills. My teams had the pleasure of publically releasing our games onto Steam, where they are available to download and play, such as Carma which has over 500 players. All of the projects and games that I have produced have been made in Unity, using C#. In my free time, and through personal projects, I have experimented with other programming languages, such as JavaScript, Python and HTML & CSS.
My passion for programming, and more specifically game development, sprouted from my love for video games. From a young age, I would always watch my brother and dad play on the Xbox classic. I grew up playing games like Simpsons Hit & Run, Tomb Raider, Star Wars Battlefront, and Halo. It's no surprise that once I started gaming on a computer that I found a love for Minecraft and Assassins Creed. In recent years, my go-to games have been Overwatch, Fortnite and GTA V. Besides gaming, I really enjoy reading manga, and watching anime. I also find joy in keeping my skills fine-tuned with hackathons and game jams.
Other than my impressive experience, I have accomplished many achievements and awards throughout my life. I have a full UK drivers license, which has been kept clean since I obtained it. Upon reaching university, I became a Student Mentor and Microsoft Student Partner, and was certified in Azure Fundamentals. Furthermore, I have an Enhanced DBS, which was needed when I was a tutor.


The How
In 2016, an unprecedented event happened: Western Australia’s coastline suffered a mass-bleaching, which means 80% of the coral became sick and vulnerable.
This matters because they could easily die off, jeopardizing ecosystems full of fish and invertebrate that need them for food and shelter.
The bleaching impacts local communities, such as the fishing industry, which affects food supplies and income. The tourism industry also takes a hit, as people stop coming when the coral’s brilliant colors are gone.
This image shows the bleaching of Australia's western coastline.
"More than 90% of the world's coral reefs will die by 2050" –Independent
Climate change has caused ocean temperatures to rise across the world, leading coral reefs on coast after coast to die. From El Niño in the Atlantic to a “warm blob” in the Pacific Ocean–waters have become unusually warm, causing the coral to become stressed.
Once stressed, they bleach, which means they release the algae from their bodies and turn white. This state leaves them susceptible to sickness, starvation and death.
But what human activities lend to this trend? Pollution is a reason the waters are warming, as the atmosphere traps more greenhouse gases and contributes them to the ocean through the rain cycle.
In fact, temperatures are rising at twice the rate they did 50 years ago, according to National Geographic. Coral is extremely sensitive to temperature change; although they prefer warm waters, even a degree and a half difference can lead to stress.
Coral mining, overfishing and digging canals are some of the top reasons the ecosystem is changing as well. These activities kill and displace a disproportionate amount of organisms, changing the makeup and regulation of the ecosystem.
The Where
Click and drag on the map below to view various locations where coral reefs are in jeopardy. Click each marker for additional information.
While coral reefs across the world have experienced bleaching, two reefs have died in the Pacific: Japan’s Sekiseishoko Reef and Jarvis Island Reef. Seventy percent of Sekiseishoko Reef is completely dead, with another 30% white from bleaching.
Jarvis Island began dying in 2015 and Sekiseishoko died earlier this year.
The Great Barrier Reef, Australia’s famous spectacle–so magnificent that it can be seen from space–is starting to die. Miles of the reef’s northern coast completely died last year due to exceedingly hot waters.
"Two-thirds of the [The Great Barrier Reef’s northern coast] were dying and are now dead" said James Cook, Coral Reef Studies, University of Australia.
And its southern regions are blanching. The extent to which the reef has experienced damaged is grave; these effects, because of pollution and overheating of the water, were not expected for another 30 years.
Another unexpected finding occurred in 2017: scientists found an unusual reef at the mouth of the Amazon River. Although coral is rarely found where fresh and salt water mix and typically do not form in rivers, the Amazon Reef defies all rules. It boasts over 3,000 miles of coral, sponges and more.
Unfortunately, it is no exception to bleaching and suffers from the effects of global warming. For a newly-found discovery, whose ecosystem is one-of-its-kind in the world, the death of this coral reef would be devastating. The Brazilian government had given oil companies the rights to drill in that space, before they discovered the reef.
The government is still debating the fate of the space.
Effects
Below are further examples of what diseased coral looks like: note the patches of white that cover each one's surface. Click the images below to explore their flickr pages, locations and tags.
The Why
But why should you care about this issue? Well, other people and groups have taken to social media to share their opinions as to why saving coral reefs is imperative. Here are a few hashtags formed to increase awareness and create change.
Hashtags and The Reason Each One Matters
-
#Wettribe:
Wettribe is an organization that actively works to reverse contaminants in the water, that support global warming down the road. Their mission is to increase awareness about how this problem destroys ecosystems, impacts our health and provides organic and harm-free alternatives for clothing, food and products. -
#ParisAgreement:
The Paris Agreement was an agreement signed in 2016 under the United Nations to deal with CO2 emissions and address climate change. This agreement is legislation that could impact the future of coral bleaching and slowly reverse the effects of pollution. President Donald Trump announced that he would remove the United States from this agreement, which under contract could only happen in November 2020. Social media users have taken to Twitter to announce their stance on their agreement and pushback in favor of it, in the hopes to stop coral bleaching. -
#OceanAcidification:
OceanAcidification is the concept that as more CO2 and greenhouse gases are trapped in the atmosphere (as a quarter of the gas we emit becomes trapped), the chemistry of the ocean changes also. In this case, the pH of the ocean has dropped, causing it to become more acidic. This harms the creatures within the seas, from the corals to fish, snails, snakes and more.


$23/35: Flag for 1541 (0), 1540 (not 0) Not used (track for drive 1) ROM-Reference: LDA $23 : $E979 STA $23 : $FF0D Previous Page / Next Page


I am a full stack developer with a passion to troubleshoot problems that occur, I like to go above and beyond and when working on code, and I leave no rock unturned, I am also an avid sports fan.
I specialize in the JavaScript Language with ES6 Syntax, I am fluent in HTML5, CSS3, React, Redux, Reactstrap, React-Bootstrap, Bootstrap, Tailwind, MaterializeCSS, Jest, and Cypress IO.
I also specialize in NodeJs, Express, MongoDB, Mongoose, SQL, SQLite, and PostgreSQL.


Top Stories
Beeston high street like Labyrinth
Trams cause chaos
Central discriminates against E-Cig smokers
Sitemap
1: Homepage
--- a) Project Description
--- b) Target Audience
--- c) Tone of the Site
--- d) Formative Testing
--- e) Colour Scheme & Fonts
--- a) Navigational Models
--- b) Paper Based Design
--- c) Computer Based Design
--- d) Wireframe
--- e) Alternative Wireframe
--- f) Internal Structure
4: Flash
--- a) Storyboard
--- b) Images
--- c) Creation
--- d) Weather Report
5: Video
--- a) Storyboard
--- b) Raw Video
--- c) Creation
--- d) News Report
6: Summary
--- a) Browser Testing
--- b) Sitemap
--- c) Credits


Headings
- src
- base
- headings.scss
Description
Sandbox uses a 1.3333typography scale starting at a base of 1.6rem. As you'll see in later sections, font-size utility classes can be used to scale components and other elements.
Naturally, the headings have the related font-size already applied. They also have a line-height of 1.5 applied to them.
Title
Title
Title
Title
Title
Title
| Element | Font Size |
|---|---|
| h1 | 3.8rem |
| h2 | 2.8rem |
| h3 | 2.1rem |
| h4, p | 1.6rem |
| h5 | 1.2rem |
| h6, small | 0.9rem |


Open browser console (press F12) to watch messages ..
Press Send Hello to send a message and get an echo from the WebSockets server.


|
Read this newsletter from your browser
[the cutting edge]
G'day Developers!
5 things to love about Dynamics CRM 2015
I've been meaning to talk about the new Microsoft Dynamics CRM 2015 for quite some time
now. After implementing it on-premises for several clients and upgrading our own
installation, I was thrilled to find that many of the long standing problems I had been
raising for years have now been solved. I was also surprised to find a nice new UI and
some cool new features to boot. Did I mention there's an iPad app?
Keep reading to find out the 5 things I love about Dynamics CRM
2015
-Adam Cogan, SSW
Got a comment for Adam?
Comment on Adam Cogan's blog
Tweet @adamcogan
Email Adam Cogan
|
New SSW TV video!
Dev Superpowers Episode 12 | Jumping into Windows Internet of Things
development by using the Universal Windows Platform
In this presentation, Chris Briggs will teach you how to jump head first into
Windows Internet of Things (IoT) development by using the Universal Windows Platform
(UWP), allowing developers to build one solution across Microsoft hardware
platforms.
Chris will show you the basics of Windows Universal app development with a focus on
building apps for devices running Windows 10 IoT Core, through to how to integrate the
solution with Windows Azure.
You will see Chris build a simple Universal app, deploy it to Raspberry Pi 2, then
expand upon the application to read input from sensors and finally establish
communication with Azure. This will demonstrate how to take a simple "hello world" thing
and make it a useful citizen of IoT.
|
Webinar: Dev Superpowers - Building an Onion Architecture with Entity Framework
Cost: No Charge
Within this presentation Brendan Richards will explore and demonstrate how features of
Entity Framework can assist in the implementation of a domain-centric,
persistence-ignorant Onion Architecture design pattern.
This talk explores techniques in avoiding spaghetti code including Onion Architecture,
the Repository Pattern, and Dependency Injection followed by how to effectively apply
these principles to your persistence layer.
Finally, Brendan will introduce SSW Data Onion – a free library developed to speed up
the day-to-day development of Data-Driven enterprise web applications using Entity
Framework Code First.
About the presenter:
|
|
Brendan Richards has been developing enterprise web applications for quite
some time across a number of platforms before finally picking .NET as the
"currently winning" choice. He Joined the team at SSW in 2012 and is now a
Solution Architect based in Brisbane.
|
July
30
Thu
|
Online - see registration page for timezone
details
|
|
Become a Full Stack Web Developer in just 9 weeks with FireBootCamp!
Special Intensive Course
Redmond, Sydney, Melbourne, Brisbane, San Francisco
The FireBootCamp program is an intensive, full-time, 9-week program. Participants learn,
work and code together for 10-12 hours per day, 5 days
a week. Apart from exercise and food breaks, the guys in the program are hands on
learning all day every day.
The program focuses on real world skills including Agile software development practices,
enterprise architecture, application life cycle,
end-to-end testing and user interface design. The technology stack covers .NET MVC,
AngularJS, TypeScript, jQuery, Windows Azure and uses Microsoft Visual Studio 2015.
July
13
Mon
|
Redmond, WA, USA
|
Oct
19
Mon
|
Sydney, NSW, Australia
Xamarin Mobile Apps Special
|
Nov
30
Mon
|
Sydney, NSW, Australia
|
|
Xamarin Hack Days around the world
Learn and love Xamarin
Xamarin is a product that allows you to use .NET/C# to create native apps that will run
across Android, iOS, and Windows.
While there is always a learning curve when using a new product, Xamarin Hack Days make
it easy for anyone who wants to learn
cross-platform development in a supportive and friendly environment.
There will be something for everyone - experienced Xamarin developers can share ideas
with other experienced developers, and
if you're a beginner then there's plenty to learn on the day. Best of all, it's totally
free.
Aug
01
Sat
|
Sydney, NSW, Australia
|
Sept
19
Sat
|
Brisbane, QLD, Australia
|
Sept
12
Sat
|
Brussels, Belgium
|
|
Sydney .NET User Group
The best place to learn .NET for free in Sydney
|
|
Sydney .NET User Group
Wednesday 15th July 2015
6:30pm - 8:30pm
Microsoft Sydney
1 Epping Rd
North Ryde NSW 2113
|
Note: location and time have changed. User group is now being held
at 6.30pm at Microsoft Sydney - 1 Epping Road, North Ryde.
"C# Is The Future" & "No More Deadlocks" – Filip Ekberg
C# Is The Future
The next version of C# is upon us so if you thought that this language was long
forgotten by its creators come along on this journey where I’ll prove you otherwise!
We will look at how C# has evolved over the years and what we might expect in the
future. Are you ready to embrace the programming language of the future?
No More Deadlocks
Do you find yourself in situations when your applications deadlock because of async and
await? Do you want to make your applications more reliable?
In this session we will talk about how to do asynchronous programming properly in .NET.
We will discover pitfalls and best practices which will help you along your way!
About the presenter:
|
|
Filip Ekberg is the author of C# Smorgasbord. Speaker. Pluralsight author.
Microsoft .NET MVP. Xamarin MVP. Works for @Invoice2Go. Swede living in
Sydney!
|
To see more details and RSVP, please visit
our Facebook
.NET Group,
our LinkedIn .NET Group
or our Meetup Group.
|
Can't make it to the Sydney .NET User Group?
Watch the Live stream!
|
|
Tune in from anywhere!
Sydney .NET User Group Live
15th July 2015
6:30PM AEST (UTC+10)
The team at SSW TV are now streaming our User Group sessions live. Each
session will also be published on the SSW TV site after the event so you can
review what was said in the sessions.
|
|
Then join us at the NerdDinner
Where nerds meet and eat
SSW and Octopus Deploy invite you to join us for pizza after the session. It's a fun and
delicious opportunity for networking with peers and sharing thoughts after the Sydney
.NET User Group presentation.
Sydney .NET User Group and NerdDinner proudly sponsored by:
|
Canberra .NET User Group
The best place to learn .NET for free in Canberra
|
|
Canberra
.NET User Group
Monday 13th July 2015
6:00pm - 8:00pm
Microsoft Canberra Branch
Level 4, 6 National Circuit,
Barton, Australia
|
"C# Is The Future" & "No More Deadlocks" – Filip Ekberg
C# Is The Future
The next version of C# is upon us so if you thought that this language was long
forgotten by its creators come along on this journey where I’ll prove you otherwise!
We will look at how C# has evolved over the years and what we might expect in the
future. Are you ready to embrace the programming language of the future?
No More Deadlocks
Do you find yourself in situations when your applications deadlock because of async and
await? Do you want to make your applications more reliable?
In this session we will talk about how to do asynchronous programming properly in .NET.
We will discover pitfalls and best practices which will help you along your way!
About the presenter:
|
|
Filip Ekberg is the author of C# Smorgasbord. Speaker. Pluralsight author.
Microsoft .NET MVP. Xamarin MVP. Works for @Invoice2Go. Swede living in
Sydney!
|
To see more details and RSVP, please visit
our Facebook
.NET Group,
our LinkedIn .NET Group or
our Meetup Group.
Canberra .NET User Group proudly sponsored by:
|
Advertise with SSW
If you would like to advertise your product or service with SSW or would like to
find out more about various advertising opportunities, please email your enquiry
to<EMAIL_ADDRESS>or call: +61
2 9953 3000.
|
|
July 2015
|
Employment
Senior SharePoint + Web Developer
SSW Brisbane
SSW is looking for an experienced and talented SharePoint developer who is passionate
about implementing awesome SharePoint solutions. This role requires great communication
skills and solid experience delivering production-ready SharePoint solutions.
- Australian Citizenship or Permanent Residency
- Commercial SharePoint 2010 and/or 2013 experience
- ASP.NET MVC using Razor and C#
- An IT degree or equivalent experience
- A keen interest in the development community including technological trends, news,
people and innovations
|
|
Chief Architect
SSW Beijing
The chief architect will be in charge of the company's daily operations, including
implementing sales plans, developing marketing plans, and leading project teams.
- Must be a strong, fluent communicator in both English and Mandarin
- A background in software engineering
- Enjoy working with a team who has strong technical skills in Microsoft
technologies
- A bachelor's degree in Computer Science, Software Engineering or Business
Management
- Strong influencing impact - the applicant should be able to engage on an executive
level
- Must be living in or willing to relocate to Beijing, China
More SSW jobs
|
|


--- layout: page title: About header-img: "img/about-bg.png" ---
I am a 25 year old personal trainer from New Jersey looking to switch to a career in the tech industry.


HAMLET 1996 A film review by Timothy Voon Copyright 1998 Timothy Voon 2 :-) :-) for the Prince of Denmark
Cast: Kenneth Branagh, Derek Jacobi, Julie Christie, Richard Briers, Nicholas Farrell, Kate Winslet, Michael Maloney, Brian Blessed, Rufus Sewell, Billy Crystal, Jack Lemmon, Charlton Heston, Gerard Depardieu, Rosemary Harris, Timothy Spall, Reece Dinsdale, Robin Williams Director: Kenneth Branagh
This version of HAMLET, will undoubtedly be remembered as the prettiest version of the Shakespearean classic. Kenneth Branagh has filled this production with more mirrors than at Tiffany's, and more rose petals than at a garden show. If you can't change the words, at least you can change the look. Not that this is bad, because décor is important in a Shakespearean play. After all, the poetry of timeless verse, needs a garden in which to sow it's timeless seed. It would take an absolute moron who can't recite lines, to destroy the language of Shakespeare, but one has to compliment Branagh and Hollywood ensemble, for delivering HAMLET with great flourish and relish.
I stood in front of the video shelf, undecided as to whether to borrow the 2 hour or 4 hour version of the feature film. Time set it's limits on my agenda and 2 got the better of 4. As I sat down, half-heartedly to watch the great play, not in the mood for Shakespeare after a weary day. Slowly, I was entranced by the lyrical spell of the dead poet. His words seemed vibrant and alive, as if calling from the grave. Hours seemed like minutes, and I began to wish I had borrowed the four-hour-for-two-days version of the play. My heart longed to hear more musings from the great courts of the Danes, where disloyal brothers murder kinsmen, to claim a crown and bed a wife, only to drive the young prince Hamlet into royal rage. For no matter how often this story is told, ‘the cat's will mew and the dog's will have their day.'
The cast includes Kenneth Branagh as an intense Hamlet, Kate Winslet as the forlorn Ophelia, Julie Christie as the unfaithful Queen Mother, Gertrude and Derek Jacobi as the murderous uncle, Claudius. With notable performances from Billy Crystal as The Gravedigger, Jack Lemmon as Marcellus, Robin Williams as Osric and many others. This is probably the most faithful version of the classic tale, and should be appreciated by all followers of the master writer. Timothy Voon e-mail<EMAIL_ADDRESS>Movie Archives http://us.imdb.com/M/reviews_by?Tim+Voon Hugues Bouclier's Movies in Melbourne http://www.labyrinth.net.au/~bouclier/week/movies.html
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Contact Form
Please fill all texts in the fields.


Sub-County Bursary Online Application
Contact Us
Gallery
Application Form
Bursaries
About Us
Home
Personal Details
First Name
Middle Name
Last Name
Birthday
Gender
Female
Male
Location
County
Country
Educational Background
Highschool Attended
Enter date before 1990-01-01:
Enter date after 2000-01-01:
Grade Attained
University
Type of Bursary
Fully-paid
Partially-paid
Reasons for Applying
Submit
Submit


Databases are essential in most applications as data needs to be stored, retrieved and queried. Databases can store all kinds of data such as application state, facts, or user credentials. There exist different types of databases on the market: some are generalist while others are specialized for certain types of use-cases, access patterns and data.
In this chapter we explore database and state management with Vert.x by diving into the implementation of the user and activity services. These services will allow us to use a document-oriented database (MongoDB) and a relational database (PostgreSQL). We will also see how to use MongoDB for authenticating users, and how to write integration tests for data-driven services.
Vert.x offers a wide range of clients to connect to data sources. These clients contain drivers to talk to servers, and may offer efficient connection management like connection pooling. This is useful to build all kinds of services, from API backed by a data source to integration services mixing data sources, messaging and APIs.
The Eclipse Vert.x project provides the following data client modules, as given in table 10.1.
| Identifier | Description |
|
|
MongoDB is a document-oriented database. |
|
|
Supports any relational database that offers a JDBC driver. |
|
|
Access PostgreSQL and MySQL relational databases through dedicated Vert.x reactive drivers. |
|
|
Redis is versatile data structure store. |
|
|
Apache Cassandra is a database tailored for very large volumes of data. |
You can find drivers for other kinds of data sources in the larger Vert.x community, that is, beyond the scope of the project at the Eclipse Foundation.
MongoDB is a popular document-oriented database. It is a good match with Vert.x since it manipulates JSON documents. Redis is an in-memory data structure store with configurable on-disk data snapshots that can be used as a cache, as a database, and as a message broker. Apache Cassandra is a multi-node, replicated database designed for storing huge amounts of data. Cassandra is well-suited for databases where size is measured in hundreds of terabytes or even petabytes. You can of course use it for just a few terabytes, but perhaps a more traditional database suffices in these cases.
Speaking of "traditional" relational databases, Vert.x can connect to anything for which there exists a JDBC driver. That being said JDBC is an older protocol based on a multi-threaded design and blocking I/O. The JDBC support in Vert.x offloads database calls to worker thread pools, and pushes results back to event-loop contexts. This is to avoid blocking event-loops since JDBC calls do block. Of course this design limits scalability as worker threads are needed, but for moderate workloads this should be fine.
Now if you use PostgreSQL or MySQL then Vert.x provides its own reactive drivers. These drivers implement the network protocols of each database servers, and they are built in a purely asynchronous fashion using Netty, the networking foundation of Vert.x. The drivers offer excellent performance, both in terms of latency and concurrent connections. They are also very stable and implement the current protocols and features of the databases. You should prefer the Vert.x reactive driver clients for PostgreSQL and MySQL, and use the JDBC client when you need to connect to other databases.
If you are looking for a solid database then PostgreSQL is probably a good bet. PostgreSQL is versatile and has been used in all sorts of small and large scale projects over the years. You can of course use it as a traditional relational database, but it also supports JSON documents as first-class objects, and geographic objects through the PostGIS extension.
Before we dive into the user profile service design and implementation with MongoDB, I would like to quickly discuss certain established idioms of enterprise Java development, and explain why in search of simplicity and efficiency the code in this chapter deviates on-purpose from supposed "best practices".
The code of the 10k steps challenge may surprise you because it does not perform object data mapping, where any data has to be mapped to some Java object model that represent the application domain (e.g., data transfer objects [DTO]). For instance some JSON data representing a pedometer update would be mapped to some DeviceUpdate Java class before any further processing is being made. Here we will directly manipulate data in JsonObject instances as they flow between HTTP, Kafka and database interfaces. We will not map, say, device update JSON data to DeviceUpdate and we will "simply" work with the JsonObject representation of that data instead.
Of course Vert.x does allow you to do data mapping from and to Java classes, but unless the object model actually contains some significant business logic or can be leveraged by some processing in a third-party library, I see little value in doing any form of data binding. I advocate such a design because:
In case of doubt always ask yourself whether you actually need an object model, or whether the data representation is good enough for the processing work that you are doing. If your object model consists of nothing but getters and setters then perhaps it’s a good sign that (at least initially) you don’t need it.
Let us now dive into using MongoDB in the user profile service.
The user profile service manages user data such as name, email, city and is also use to authenticate a user against login / password credentials. This service is used by the services that need to retrieve and correlate data against user information.
The user service makes use of MongoDB for 2 purposes:
MongoDB is a good fit because it is a document database, so each user can be represented as a document. We are going to use the vertx-mongo-client module to connect to MongoDB instances, and we are also going to use the vertx-auth-mongo module for doing authentication.
The vertx-auth-mongo module is a "turn-key" solution for doing user authentication on top of a MongoDB database, as it manages all the intricacies of properly storing and retrieving credentials. It implements the common authentication interface of module vertx-auth-common. It especially deals with storing cryptographic hashes of passwords with a salt value, because storing actual passwords is never a good idea. According to the conventions defined in the vertx-auth-mongo module, there is a document for each user in the target database with the following entries:
username
: s gsrint lkt vry eearmuns,salt
: s ndmaro yrcz itgnsr ucxh rk useerc krb osawrpsd,password
: z igntrs syvm xl cuotimpng rqv SHB-512 cadb vltm bxr tluaac aorsswpd ahdf urx salt
ulvae,roles
: cn rarya lx tngsris gfiiednn lesro (o.p., nr"stao"rtimida),permissions
: sn yrraa lx trsnsig egifdinn rmspisoiens (k.q., aeccc"_n_as"sbeat).In our case we are not going to use roles and permissions since all users will be equal, and these entries will be empty arrays. We will not have to deal with the subtleties of handling salts and password hashing as this is taken care of by the authentication module.
While this data model is prescribed by vertx-auth-mongo, nothing precludes us from adding more fields to the documents that represent users. We thus add the following entries:
city
: s nirtgs tel qrx yzvt jsur,deviceId
: s gstrni klt rou reeptdoem ceevdi driinefiet,email
: s risngt etl grx toch ameli,makePublic
: s anboelo rk inadecit ewhert pvr vaht awtns kr eapapr nj cliubp nrankgis xt nxr.We also enforce 2 integrity constraints with MongoDB indexes: both username and deviceId must be unique to a document. This avoids duplicate user names as well as 2 users having the same device. This will interestingly pose to be a correctness challenge when registering new users because we will not be able to use any transaction mechanism, so we will need to rollback partial data inserts when the deviceId uniqueness constraint will prevent a duplicate insert.
Let us now see how to use both the Vert.x MongoDB client and the Vert.x authentication support.
The UserProfileApiVerticle class exposes the HTTP API for the user profile service. It holds 2 important fields:
mongoClient
lx vgrg MongoClient
rk necncot xr s WvnvhNR servre, bncauthProvider
xl rqkg MongoAuth
rk mrorefp thaaniteiotnuc gnusi WvxdnUR.We initialize these fields from the verticle initialization method rxStart (since we use RxJava), as given in listing 10.1.
mongoClient = MongoClient.createShared(vertx, mongoConfig()); #1 JsonObject authConfig = new JsonObject(); #2 authProvider = MongoAuth.create(mongoClient, authConfig);
The authentication provider piggy-backs onto the MongoDB client instance, which is configured as in listing 10.2. We pass empty configuration options for the authentication provider as we follow the conventions of the Vert.x MongoDB authentication module.
private JsonObject mongoConfig() { return new JsonObject() .put("host", "localhost") #1 .put("port", 27017) .put("db_name", "profiles"); #2 }
Since we expose a HTTP API, we use a Vert.x web router to configure the various routes to be handled by the service, given in listing 10.3.
Router router = Router.router(vertx); BodyHandler bodyHandler = BodyHandler.create(); router.post().handler(bodyHandler); router.put().handler(bodyHandler); router.post("/register") .handler(this::validateRegistration) #1 .handler(this::register); router.get("/:username").handler(this::fetchUser); router.put("/:username").handler(this::updateUser); router.post("/authenticate").handler(this::authenticate); router.get("/owns/:deviceId").handler(this::whoOwns);
We can note that we use 2 chained handlers for the registration, as we use a first handler for data validation, and a second handler for the actual processing logic. But what is exactly in the validation logic?
Registration is a critical step, so we must ensure that data is valid. We must check that the incoming data (a JSON document) contains all required fields, and that they are all valid. For instance we need to check that an email is actually an email, and that a username is not empty and does not contain unwanted characters.
The validateRegistration method of listing 10.4 delegates to helper methods anyRegistrationFieldIsMissing and anyRegistrationFieldIsWrong to perform validation.
private void validateRegistration(RoutingContext ctx) { JsonObject body = jsonBody(ctx); if (anyRegistrationFieldIsMissing(body) || anyRegistrationFieldIsWrong(body)) { ctx.fail(400); #1 } else { ctx.next(); #2 } }
When any validation steps fails then we respond with a 400 HTTP status code, else we call the next handler, which in our case will be the register method. The implementation of method anyRegistrationFieldIsMissing is quite simple as we check that the provided JSON document contains the required fields, as shown in listing 10.5.
private boolean anyRegistrationFieldIsMissing(JsonObject body) { return !(body.containsKey("username") && #1 body.containsKey("password") && body.containsKey("email") && body.containsKey("city") && body.containsKey("deviceId") && body.containsKey("makePublic")); }
Method anyRegistrationFieldIsWrong delegates checks to regular expressions with the implementation of listing 10.6.
private final Pattern validUsername = Pattern.compile("\\w[\\w+|-]*"); #1 // (...) private boolean anyRegistrationFieldIsWrong(JsonObject body) { return !validUsername.matcher(body.getString("username")).matches() || #2 !validEmail.matcher(body.getString("email")).matches() || body.getString("password").trim().isEmpty() || #3 !validDeviceId.matcher(body.getString("deviceId")).matches(); }
The validDeviceId regular expression is the same as validUsername. Validating an email address (validEmail) is a more sophisticated regular expression. We chose to use one of the safe regular expressions from [OWASP] for that purpose.
Now that we have validated data, it is time to actually register users.
Inserting a new user in the database requires 2 steps:
Since this is a 2 steps data insert and we cannot use any transaction management facility, we need to care about data integrity ourselves, as shown in figure 10.1.
Fortunately RxJava makes the error management declarative, so we won’t have to deal with nested conditionals of asynchronous operations, which would be complicated to do with callbacks or promises / futures.
The register method starts with extracting the JSON payload from the HTTP request, then the username and password of the user to create, as shown in listing 10.7.
private void register(RoutingContext ctx) { JsonObject body = jsonBody(ctx); #1 String username = body.getString("username"); String password = body.getString("password"); authProvider .rxInsertUser(username, password, emptyList(), emptyList()) #2 // (...) }
Remember that register is called after validation, so we expect the JSON data to be good. We pass the authentication provider the username, password, and 2 empty lists for roles and permissions. It then populates the database with a new document. We then have to run a query to update the newly-created document, and append new entries. The MongoDB query is given in listing 10.8 and is represented as a JSON object.
JsonObject extraInfo = new JsonObject() .put("$set", new JsonObject() #1 .put("email", body.getString("email")) .put("city", body.getString("city")) .put("deviceId", body.getString("deviceId")) .put("makePublic", body.getBoolean("makePublic")));
We must thus chain the rxInsertUser operation with a MongoDB update query, knowing that rxInsertUser returns a Single<String> where the value is the identifier of the new document. Listing 10.9 has the complete user addition processing with RxJava.
authProvider .rxInsertUser(username, password, emptyList(), emptyList()) #1 .flatMapMaybe(docId -> insertExtraInfo(extraInfo, docId)) #2 .ignoreElement() .subscribe( () -> completeRegistration(ctx), #3 err -> handleRegistrationError(ctx, err)); #4
The flatMapMaybe operator allows us to chain the 2 queries. Method insertExtraInfo is given in listing 10.10 and returns a MaybeSource, because finding and updating a document may not hold a result if no matching document was found.
Note that the update query can fail, for instance because another user has already registered a device with the same identifier. In this case we need to manually rollback and remove the document that was created by the authentication provider, because otherwise we would have an incomplete document in the database. Listing 10.11 holds the implementation of the deleteIncompleteUser method.
private boolean isIndexViolated(Throwable err) { return err.getMessage().contains("E11000"); #1 } private MaybeSource<? extends JsonObject> deleteIncompleteUser(JsonObject query, Throwable err) { if (isIndexViolated(err)) { return mongoClient .rxRemoveDocument("user", query) #2 .flatMap(del -> Maybe.error(err)); #3 } else { return Maybe.error(err); #4 } }
We need to rely on a technical code in an exception message to distinguish between index violation errors and other types of errors. In the first case the previous data has to be removed because we want to deal with it and recover, in the second case this is another error and we cannot do much so we propagate it.
Finally the handleRegistrationError given in listing 10.12 method needs to inspect the error to respond with the appropriate HTTP status code.
private void handleRegistrationError(RoutingContext ctx, Throwable err) { if (isIndexViolated(err)) { logger.error("Registration failure: {}", err.getMessage()); ctx.fail(409); #1 } else { logger.error("Woops", err); ctx.fail(500); #2 } }
It is important to notify the requester whether the request failed because the username or device identifier has already been taken, or whether it failed due to some technical error. In one case the error is the fault of the requester, in the other case the service is the culprit and the requester can try again later.
Authenticating a user against a username and password is very simple. All we need to do is querying the authentication provider, which returns a io.vertx.ext.auth.User instance on success. In our case we are not interested in querying permissions or roles, so all we want to do is check that authentication succeeded.
Assuming that a HTTP POST request sent to /authenticate has a JSON body with username and password fields, we can perform the authentication request as in listing 10.13.
private void authenticate(RoutingContext ctx) { authProvider.rxAuthenticate(jsonBody(ctx)) #1 .subscribe( user -> completeEmptySuccess(ctx), err -> handleAuthenticationError(ctx, err)); } private void completeEmptySuccess(RoutingContext ctx) { ctx.response().setStatusCode(200).end(); #2 } private void handleAuthenticationError(RoutingContext ctx, Throwable err) { logger.error("Authentication problem {}", err.getMessage()); ctx.response().setStatusCode(401).end(); #3 }
The result of an authentication request is a User or an exception if it failed. Depending on the outcome we end the HTTP request with a 200 or 401 status code.
HTTP GET requests to /username must return the data associated to that user (e.g., /foo, /bar, etc). To do that we need to prepare a MongoDB query, and return the data as a JSON response.
We need a MongoDB "find" query to locate a user document. To do that we need 2 JSON documents:
username
ifeld lx ogr esaadtab suncoemdt, nzpThe code to perform such a query is in listing 10.14.
JsonObject query = new JsonObject() .put("username", username); #1 JsonObject fields = new JsonObject() .put("_id", 0) #2 .put("username", 1) #3 .put("email", 1) .put("deviceId", 1) .put("city", 1) .put("makePublic", 1); mongoClient .rxFindOne("user", query, fields) #4 .toSingle() .subscribe( json -> completeFetchRequest(ctx, json), err -> handleFetchError(ctx, err));
It is important to specify which fields should be part of the response, and be explicit about it. In our case we don’t want to reveal the document identifier so we set it to 0 in the fields document. We also explicitly list the fields that we want to be returned with 1 values. This also ensures that other fields like the password and salt values from the authentication do not get accidentally revealed.
Listing 10.15 shows the 2 methods to complete the fetch request and HTTP response.
private void completeFetchRequest(RoutingContext ctx, JsonObject json) { ctx.response() .putHeader("Content-Type", "application/json") .end(json.encode()); #1 } private void handleFetchError(RoutingContext ctx, Throwable err) { if (err instanceof NoSuchElementException) { #2 ctx.fail(404); } else { fail500(ctx, err); } }
It is important to properly deal with the error cases and distinguish between a non-existing user and a technical error. Let us now see the case of updating a user.
Updating a user data shares similarities with fetching data, as we need 2 JSON documents again: one to match documents, and one to specify what fields need to be updated. Listing 10.16 shows the corresponding code.
JsonObject query = new JsonObject().put("username", username); #1 JsonObject updates = new JsonObject(); if (body.containsKey("city")) { #2 updates.put("city", body.getString("city")); } if (body.containsKey("email")) { updates.put("email", body.getString("email")); } if (body.containsKey("makePublic")) { updates.put("makePublic", body.getBoolean("makePublic")); } if (updates.isEmpty()) { #3 ctx.response().setStatusCode(200).end(); return; } updates = new JsonObject().put("$set", updates); #4 mongoClient .rxFindOneAndUpdate("user", query, updates) #5 .ignoreElement() .subscribe( () -> completeEmptySuccess(ctx), err -> handleUpdateError(ctx, err));
Since the update request is a JSON document coming from a HTTP request, there is always a possibility of an external attack if we are not careful. We could indeed imagine that a malicious user crafts a JSON document in the request with updates to the password or username, and this is why we test for the presence of each allowed field in updates: city, email and makePublic. We then create a JSON document with updates just for these fields rather than reusing the JSON document received over HTTP, and make an update request to the Vert.x MongoDB client.
We have just covered typical usage of MongoDB in Vert.x, as well as how to use it for authentication purposes. Let us now move on to PostgreSQL and the activity service.
The activity service stores all steps updates as they are received from pedometers. It is a service that reacts to new step update events (to store data), and it can be queried by other services to get step counts for a given device on a given day, month or year.
The activity services uses PostgreSQL to store activity data after device updates have been accepted by the ingestion service. PostgreSQL is well-suited because the SQL query language makes it easy to compute aggregates such as step counts for a device on a given month.
The service is split into 2 independent verticles:
EventsVerticle
seitnsl tle iconignm tviacity pesdtau oteo Dlzoz nqc rnxd oetsrs shrs rjxn rgk bdeataas, hlweiActivityApiVerticle
eexosps c HAAL CLJ tlv niruyqge ytviaict srqz.We could have put all the code on a single verticle, but this decoupling renders the code more manageable as each verticle has a well-defined purpose. EventsVerticle performs writes to the database, while ActivityApiVerticle performs the read operations.
The data model is not terribly complex and fits in a single relation stepevent. The SQL instructions for creating the stepevent table are given in listing 10.17.
CREATE TABLE IF NOT EXISTS stepevent ( device_id VARCHAR, device_sync BIGINT, sync_timestamp timestamptz, #1 steps_count INTEGER, PRIMARY KEY (device_id, device_sync) #2 );
The primary key uniquely identifies an activity update based on a device identifier (device_id) and a synchronization counter from the device (device_sync). The timestamp of the event is recorded (sync_timestamp), and finally the number of steps is stored (steps_count).
If you come from a background with a heavy usage of object-relational mappers (ORMs) then you may be surprised by the database schema above, and especially the fact that it uses a composite primary key rather than some auto-incremented number. You may want to first consider the proper design of your relational model with respect to normal forms, and only then see how to handle data in your code, be it with collections and/or objects that reflect the data.
The vertx-pg-client module contains the PgPool interface that models a pool of connections to a PostgreSQL server, where each connection can be reused for subsequent queries. PgPool is your main access point in the client to perform SQL queries. Listing 10.18 shows how to create a PostgreSQL connection pool.
PgPool pgPool = PgPool.pool(vertx, PgConfig.pgConnectOpts(), new PoolOptions()); #1 // (...) public static PgConnectOptions pgConnectOpts() { #2 return new PgConnectOptions() .setHost("localhost") .setDatabase("postgres") .setUser("postgres") .setPassword("vertx-in-action"); }
The pool creation requires a Vert.x context, a set of connection options such as the host, database and password, and pool options. The pool options can be tuned to set the maximum number of connections as well as the size of the waiting queue, but default values are fine here.
The pool object is then used to perform queries to the database as we are now going to see.
The EventsVerticle is in charge of listening to Kafka records on the incoming.steps topic where each record is an update received from a device through the ingestion service. For each record it must:
daily.step.updates
Qocsl opitc.This is illustrated in figure 10.2.
These steps are modeled by the RxJava pipeline defined in listing 10.19.
eventConsumer .subscribe("incoming.steps") #1 .toFlowable() .flatMap(this::insertRecord) #2 .flatMap(this::generateActivityUpdate) #3 .flatMap(this::commitKafkaConsumerOffset) #4 .doOnError(err -> logger.error("Woops", err)) .retryWhen(this::retryLater) .subscribe();
This RxJava pipeline is reminiscent of those we have seen earlier about the messaging and eventing stack, as we compose 3 asynchronous operations. This pipeline reads from Kafka, inserts database records (insertRecord), produces a query to write to Kafka (generateActivityUpdate) then commits it (commitKafkaConsumerOffset).
The SQL query to insert a record is given in listing 10.21.
static String insertStepEvent() { return "INSERT INTO stepevent VALUES($1, $2, current_timestamp, $3)"; #1 }
Vert.x does not prescribe any object-relational mapping tool. Doing plain SQL is actually a great option, but if you want to abstract your code from the particularities of databases and use an API to build your queries rather than using strings then I recommend looking at jOOQ [JOOQ]. You can even find a Vert.x / jOOQ integration module in the community!
We use a class with static methods to give SQL queries as it is more convenient than plain string constants in our code. The query will be used as a prepared statement, where values prefixed by a $ symbol will be taken from a tuple of values. Since we use a prepared statement these values are safe from SQL injection attacks.
Method insertRecord is called for each new Kafka record, and the method body is given in listing 10.21.
JsonObject data = record.value(); #1 Tuple values = Tuple.of( #2 data.getString("deviceId"), data.getLong("deviceSync"), data.getInteger("stepsCount")); return pgPool .rxPreparedQuery(insertStepEvent(), values) #3 .map(rs -> record) #4 .onErrorReturn(err -> { #5 if (duplicateKeyInsert(err)) { return record; } else { throw new RuntimeException(err); } }) .toFlowable();
We first extract the JSON body from the record, then prepare a tuple of values to pass as parameters to the SQL query of listing 10.20. The result of the query is a row set, but since this is not a SELECT query we do not care about the result. Instead we simply re-map the result with the original Kafka record value, so the generateActivityUpdate method can re-use it.
The onErrorReturn operator allows us to handle duplicate inserts gracefully. It is possible that after a service restart we end up replaying some Kafka events that we had already processed, so the INSERT queries will fail since that would result in entries with duplicate primary keys. Method duplicateKeyInsert from listing 10.22 shows how to distinguish between a duplicate key error and another technical error.
private boolean duplicateKeyInsert(Throwable err) { return (err instanceof PgException) && "23505".equals(((PgException) err).getCode()); #1 }
We again have to search for some technical error code in the exception message, and when it corresponds to a PostgreSQL duplicate key error then onErrorReturn puts the original Kafka record in the pipeline rather than let an error be propagated.
The next step in the RxJava processing pipeline after a record has been inserted is to query the database to get how many steps have been done on the current day. This is then used to prepare a new Kafka record and push it to the daily.step.updates Kafka topic.
The SQL query corresponding to that operation is given by the stepsCountForToday method from listing 10.23.
static String stepsCountForToday() { return "SELECT current_timestamp, coalesce(sum(steps_count), 0) FROM stepevent WHERE " + #1 "(device_id = $1) AND" + "(date_trunc('day', sync_timestamp) = date_trunc('day', current_timestamp))"; #2 }
This request computes the sum (or zero) of the steps done on the current day for a give device identifier. Listing 10.24 provides the implementation of the generateActivityUpdate method, picking the original Kafka record forwarded by method insertRecord.
String deviceId = record.value().getString("deviceId"); #1 LocalDateTime now = LocalDateTime.now(); String key = deviceId + ":" + now.getYear() + "-" + now.getMonth() + "-" + now.getDayOfMonth(); #2 return pgPool .rxPreparedQuery(stepsCountForToday(), Tuple.of(deviceId)) #3 .map(rs -> rs.iterator().next()) #4 .map(row -> new JsonObject() #5 .put("deviceId", deviceId) .put("timestamp", row.getTemporal(0).toString()) .put("stepsCount", row.getLong(1))) .flatMap(json -> updateProducer.rxSend(KafkaProducerRecord.create("daily.step.updates", key, json))) #6 .map(rs -> record) .toFlowable();
This code shows us how to manipulate rows following a SELECT query. The result of a query is RowSet, materialized here by the rs argument in the first map operator, and which can be iterated row-by-row. Since the query returns a single row, we can directly access the first and single row by calling next on the RowSet iterator. We then access the row elements by type and index to build a JsonObject that makes the Kafka record sent to the daily.step.updates topic.
The ActivityApiVerticle class exposes the HTTP API for the activity service, all routes lead to SQL queries. We are not going to show all of them so let us just focus on monthly steps for a device, handled through HTTP GET requests to /:deviceId/:year/:month. The SQL query is given in listing 10.25.
static String monthlyStepsCount() { return "SELECT sum(steps_count) FROM stepevent WHERE" + "(device_id = $1) AND" + "(date_trunc('month', sync_timestamp) = $2::timestamp)"; #1 }
The stepsOnMonth method is in listing 10.26 and performs the SQL query based on the year and month path parameters.
private void stepsOnMonth(RoutingContext ctx) { try { String deviceId = ctx.pathParam("deviceId"); LocalDateTime dateTime = LocalDateTime.of( Integer.parseInt(ctx.pathParam("year")), Integer.parseInt(ctx.pathParam("month")), 1, 0, 0); Tuple params = Tuple.of(deviceId, dateTime); #1 pgPool.rxPreparedQuery(SqlQueries.monthlyStepsCount(), params) .map(rs -> rs.iterator().next()) .subscribe( row -> sendCount(ctx, row), #2 err -> handleError(ctx, err)); #3 } catch (DateTimeException | NumberFormatException e) { #4 sendBadRequest(ctx); } }
The query result is again a RowSet, and we know by the SQL query that only 1 row can be returned, so we use the map operator to extract it. The sendCount method sends the data as a JSON document, while the handleError method produces a HTTP 500 error. When a year or month URL parameter is not a number or does not result in a valid date then sendBadRequest produces a HTTP 400 to let the request know of a mistake.
It is now time to move on to integration testing strategies. This will also show you some other data client methods such as SQL batch queries when we will have to pre-populate a PostgreSQL database.
Testing the user profile service involves issuing HTTP requests to the corresponding API. The activity service has 2 facets: one that also involves the HTTP API, and one that involves crafting Kafka events and observing the effects in terms of persisted state and produced events.
The user profile tests rely on issuing HTTP request that impact the service state and the database (e.g., creating a user) then issuing further HTTP requests to perform some assertions, as illustrated in figure 10.3.
The integration tests rely again on TestContainers as we need to have a MongoDB instance running. Once we have the container running, we need to prepare the MongoDB database to be in a clean state before we run any test. This is important to ensure that a test cannot be affected by data left over by a previous test execution. The setup method of class IntegrationTest performs the test preparation as found in listing 10.27.
@BeforeEach void setup(Vertx vertx, VertxTestContext testContext) { JsonObject mongoConfig = new JsonObject() .put("host", "localhost") .put("port", 27017) .put("db_name", "profiles"); mongoClient = MongoClient.createShared(vertx, mongoConfig); mongoClient .rxCreateIndexWithOptions("user", new JsonObject().put("username", 1), #1 new IndexOptions().unique(true)) .andThen(mongoClient.rxCreateIndexWithOptions("user", new JsonObject().put("deviceId", 1), new IndexOptions().unique(true))) #2 .andThen(dropAllUsers()) #3 .flatMapSingle(res -> vertx.rxDeployVerticle(new UserProfileApiVerticle())) .subscribe( ok -> testContext.completeNow(), testContext::failNow); }
We first connect to the MongoDB database, then ensure we have 2 indexes for the username and deviceId fields. We then need to remove all existing documents from the profiles database (see listing 10.28), and deploy an instance of the UserProfileApiVerticle verticle before successfully completing the initialization phase.
The IntegrationTest class provides different test cases of operations that are expected to succeed as well as operations that are expected to fail. RestAssured is used to write the test specifications of the HTTP requests, like in listing 10.29.
@Test @DisplayName("Failing at authenticating an unknown user") void authenticateMissingUser() { JsonObject request = new JsonObject() #1 .put("username", "Bean") .put("password", "abc"); with() .spec(requestSpecification) .contentType(ContentType.JSON) .body(request.encode()) .post("/authenticate") .then() .assertThat() .statusCode(401); #2 }
The authenticateMissingUser method checks that authenticating against invalid credentials results in a HTTP 401 status code. Another example is the test of listing 10.30 where we check what happens when we attempt registering a user twice.
given() .spec(requestSpecification) .contentType(ContentType.JSON) .accept(ContentType.JSON) .body(basicUser().encode()) #1 .when() .post("/register") .then() .assertThat() .statusCode(200); #2 given() .spec(requestSpecification) .contentType(ContentType.JSON) .accept(ContentType.JSON) .body(basicUser().encode()) .when() .post("/register") .then() .assertThat() .statusCode(409); #3
Note that we could also peek into database and check the data that is being stored after each action. Since we need to cover all functional cases of the HTTP API it is more straightforward to focus on just the HTTP API in the integration tests. Now there are cases where an API on top of a database may not expose you to some important effects on the stored data, and in these cases you will need to connect to the database to do some further assertions.
The case of testing the activity service API is quite similar to that of the user profile service, except that we use PostgreSQL instead of MongoDB.
We first need to ensure the data schema is defined as in listing 10.17. To do that the SQL script in init/postgres/setup.sql is run automatically when the PostgreSQL container starts. This works because the container image specifies that any SQL script found in /docker-entrypoint-initdb.d/ will be run when it starts, and the Docker Compose file that we use mounts init/postgres to /docker-entrypoint-initdb.d/ so the SQL file is available in the container.
Once the database has been prepared with some pre-defined data, we issue HTTP requests to perform assertions as shown in figure 10.4.
We again rely on TestContainers to start a PostgreSQL server for us, then rely on the test setup method to prepare the data as in listing 10.31.
String insertQuery = "INSERT INTO stepevent VALUES($1, $2, $3::timestamp, $4)"; #1 LocalDateTime now = LocalDateTime.now(); List<Tuple> data = Arrays.asList( #2 Tuple.of("123", 1, LocalDateTime.of(2019, 4, 1, 23, 0), 6541), Tuple.of("123", 2, LocalDateTime.of(2019, 5, 20, 10, 0), 200), Tuple.of("123", 3, LocalDateTime.of(2019, 5, 21, 10, 10), 100), Tuple.of("456", 1, LocalDateTime.of(2019, 5, 21, 10, 15), 123), Tuple.of("123", 4, LocalDateTime.of(2019, 5, 21, 11, 0), 320), Tuple.of("abc", 1, now.minus(1, ChronoUnit.HOURS), 1000), Tuple.of("def", 1, now.minus(2, ChronoUnit.HOURS), 100), Tuple.of("def", 2, now.minus(30, ChronoUnit.MINUTES), 900), Tuple.of("abc", 2, now, 1500) ); PgPool pgPool = PgPool.pool(vertx, PgConfig.pgConnectOpts(), new PoolOptions()); pgPool.rxQuery("DELETE FROM stepevent") #3 .flatMap(rows -> pgPool.rxPreparedBatch(insertQuery, data)) #4 .ignoreElement() .andThen(vertx.rxDeployVerticle(new ActivityApiVerticle())) #5 .ignoreElement() .andThen(Completable.fromAction(pgPool::close)) #6 .subscribe(testContext::completeNow, testContext::failNow);
Here we want a database with a data set that we control with activities for devices 123, 456, abc and def at various points in time. For instance device 123 recorded 320 steps on 2019/05/21 at 11:00, and that was the 4th time the device made a successful synchronization with the backend. We can then perform checks against the HTTP API like in listing 10.32 where we check the number of steps for device 123 in May 2019.
JsonPath jsonPath = given() .spec(requestSpecification) .accept(ContentType.JSON) .get("/123/2019/05") #1 .then() .assertThat() .statusCode(200) .extract() .jsonPath(); assertThat(jsonPath.getInt("count")).isEqualTo(620); #2
The activity HTTP API is the "read-only" part of the service, so let us now see the other part of the service.
The technique to test the Kafka event processing part of the EventsVerticle is very similar to what we did in the previous chapter: we are going to send some Kafka records and then observe what Kafka records the service produces. By sending multiple step updates for a given device, we should observe that the service produces updates that accumulate the steps on the current day. Since the service both consumes and produces Kakfa records that reflect the current state of the database we won’t need to perform SQL queries, observing that correct Kafka record are being produced is sufficient. Figure 10.5 provides an overview of how the test is being made.
The integration test class (EventProcessingTest) again uses TestContainers to start the required services: PostgreSQL, Apache Kafka and Apache Zookeeper. Before any test is run, we must start from a clean state by using the test preparation code of listing 10.33.
consumer = KafkaConsumer.create(vertx, KafkaConfig.consumer("activity-service-test-" + System.currentTimeMillis())); producer = KafkaProducer.create(vertx, KafkaConfig.producer()); KafkaAdminClient adminClient = KafkaAdminClient.create(vertx, KafkaConfig.producer()); PgPool pgPool = PgPool.pool(vertx, PgConfig.pgConnectOpts(), new PoolOptions()); pgPool.rxQuery("DELETE FROM stepevent") #1 .flatMapCompletable(rs -> adminClient.rxDeleteTopics(Arrays.asList("incoming.steps", "daily.step.updates"))) #2 .andThen(Completable.fromAction(pgPool::close)) #3 .onErrorComplete() .subscribe( testContext::completeNow, testContext::failNow);
We need to ensure that the PostgreSQL database is empty, and that the Kafka topics that we use to receive and send events are deleted. We can then focus on the test method, where we will send 2 step updates for device 123. Before that we must first subscribe to the daily.step.updates Kafka topic where the EventsVerticle class will send Kafka records. Listing 10.34 shows the first part of the test case.
consumer.subscribe("daily.step.updates") .toFlowable() .skip(1) #1 .subscribe(record -> { #2 JsonObject json = record.value(); testContext.verify(() -> { #3 assertThat(json.getString("deviceId")).isEqualTo("123"); assertThat(json.containsKey("timestamp")).isTrue(); assertThat(json.getInteger("stepsCount")).isEqualTo(250); }); testContext.completeNow(); }, testContext::failNow);
Since we send 2 updates, we skip the emitted record and only perform assertions on the second one, as it should reflect the steps sum of the 2 updates. This code alone is waiting for events to be produced, so we now need to deploy the EventsVerticle and send the 2 updates as in listing 10.35.
vertx .rxDeployVerticle(new EventsVerticle()) #1 .flatMap(id -> { #2 JsonObject steps = new JsonObject() .put("deviceId", "123") .put("deviceSync", 1L) .put("stepsCount", 200); return producer.rxSend(KafkaProducerRecord.create("incoming.steps", "123", steps)); }) .flatMap(id -> { #3 JsonObject steps = new JsonObject() .put("deviceId", "123") .put("deviceSync", 2L) .put("stepsCount", 50); return producer.rxSend(KafkaProducerRecord.create("incoming.steps", "123", steps)); }) .subscribe(ok -> { }, testContext::failNow);
The test completes as EventsVerticle properly sends correct updates to the daily.step.updates Kafka topic. We can again note how RxJava allows us to compose asynchronous operations in a declarative fashion, and ensure error processing is clearly identified. We have essentially 2 RxJava pipelines here and any error causes the test context to fail.
There is a tiny vulnerability window for this test to fail in case the first update is sent before midnight and the second right after midnight, in which case the second event will not be a sum of the steps in the 2 events. This is very unlikely to happen since the 2 events will be emitted a few milliseconds appart, but still, it could happen!
Speaking of event streams, the next chapter will focus on the topic of advanced event processing services with Vert.x.
[OWASP] The Open Web Application Security Project (OWASP). OWASP Validation Regex Repository. Retrieved November 2019. www.owasp.org/index.php/OWASP_Validation_Regex_Repository
[Neward06] Ted Neward. The Vietnam of Computer Science. Retrieved November 2019. blogs.tedneward.com/post/the-vietnam-of-computer-science/
[DTO] Martin Fowler. Data Transfer Object. Retrieved November 2019. martinfowler.com/eaaCatalog/dataTransferObject.html
[JOOQ] Data Geekery GmbH. jOOQ. Retrieved November 2019. www.jooq.org/


Across
1)
My parents are going for a concert
tonight
.
2)
Henry
just
completed his homework.
3)
Your answer is
partially
correct.
4)
The carpenter has
already
fixed the broken chair.
Down
1)
Alia and Ria
often
study together.
2)
The children were playing
happily
in the garden.
3)
Mom
rarely
gets up early on weekends.
4)
Alex
politely
asked the teacher if he could enter the classroom.


DEAR FRIEND, I AM MR. ADAMA J.K KAMBORE, THE BILL AND EXCHANGE MANAGER AT THE FOREIGN EXCHANGE DEPARTMENT OF INTERNATIONAL BANK OF BURKINA (IBB) OUAGADOUGOU IN BURKINA FASO.  REPLY ONLY TO MY PRIVATE E-MAIL<EMAIL_ADDRESS>------------------------------------ THE FUND HAS BEEN IN HIS ACCOUNT WITH THE BANK WITHOUT ANY CLAIM OF THE FUND IN OUR CUSTODY EITHER FROM HIS FAMILY OR RELATION BEFORE OUR DISCOVERY TO THIS DEVELOPMENT. ALTHOUGH PERSONALLY, I KEPT THIS INFORMATION SECRET WITHIN MYSELF TO ENABLE THE WHOLE PLANS AND IDEA BE PROFITABLE AND SUCCESSFUL DURING THE TIME OF EXECUTION. THE AMOUNT INVOLVED IS USD$10.500,000 (TEN MILLION FIVE HUNDRED THOUSAND U.S DOLLARS.)    ON SMOOTH CONCLUSION OF THIS TRANSACTION, YOU WILL BE ENTITLED TO 40% OF THE TOTAL SUM WHILE 60% WILL BE FOR ME.     PLEASE REPLY TO MY PRIVATE E-MAIL<EMAIL_ADDRESS>BY INDICATING YOUR BANK ------------------------------------- ACCOUNT DETALS WHERE THE FUND WILL BE TRANSFERED TO.  BEST REGARD, MR. ADAMA J.K KAMBORE. _________________________________________________________________ Don't just search. Find. Check out the new MSN Search! http://search.msn.click-url.com/go/onm00200636ave/direct/01/


How about a game of tic-tac-toe?
Instructions
Click inside the grid to play...
You:
Computer:
Current status:
Your turn
Result:
...
RESET


//#- //#excludeif true
Sorry, but the page is still under construction....
Visit us later, please...


Chisholm Alley
[30, 40]
|
the Husted Building
[31, 40]
|
Landsey Grove
[32, 40]
|
Walling Bank
[33, 40]
|
McGarth Walk Fire Station
[34, 40]
|
Garner Drive
[35, 40]
|
a factory
[36, 40]
|
Bryan Place Fire Station
[37, 40]
|
Club Traves
[38, 40]
|
wasteland
[39, 40]
|
Staddon Library
[30, 41]
|
Wakley Drive
[31, 41]
|
Horsford Lane
[32, 41]
|
Martha General Hospital
[33, 41]
|
the Sands Building
[34, 41]
|
a factory
[35, 41]
|
St Eligius's Church
[36, 41]
|
a warehouse
[37, 41]
|
Wine Place
[38, 41]
|
Club Swyer
[39, 41]
|
Hemore Auto Repair
[30, 42]
|
Stovin Row
[31, 42]
|
a warehouse
[32, 42]
|
Ephrem General Hospital
[33, 42]
|
Ray Library
[34, 42]
|
Mountstephens Bank
[35, 42]
|
a cemetery
[36, 42]
|
Mogg Square
[37, 42]
|
a warehouse
[38, 42]
|
Shipton Crescent
[39, 42]
|
a junkyard
[30, 43]
|
the Swinnerton Monument
[31, 43]
|
Cuss Place
[32, 43]
|
Rolls Cinema
[33, 43]
|
Auston Auto Repair
[34, 43]
|
the Axworthy Hotel
[35, 43]
|
St Callistus's Church
[36, 43]
|
Haines Square
[37, 43]
|
a factory
[38, 43]
|
Wicksted Avenue
[39, 43]
|
the Pitter Building
[30, 44]
|
wasteland
[31, 44]
|
Ackland Mall
[32, 44]
|
Ackland Mall
[33, 44]
|
Huddy Drive
[34, 44]
|
Mallack Avenue
[35, 44]
|
Moseley Plaza Police Dept
[36, 44]
|
a carpark
[37, 44]
|
wasteland
[38, 44]
|
Crape Bank
[39, 44]
|
Woollacot Boulevard Fire Station
[30, 45]
|
Club Kick
[31, 45]
|
Ackland Mall
[32, 45]
|
Ackland Mall
[33, 45]
|
Michaud Walk Fire Station
[34, 45]
|
Leigh Way
[35, 45]
|
Beare Avenue
[36, 45]
|
Killinger Walk
[37, 45]
|
Rawles Bank
[38, 45]
|
Venables Row
[39, 45]
|
Cort Row
[30, 46]
|
St Blaise's Church
[31, 46]
|
Chaffe Cinema
[32, 46]
|
a junkyard
[33, 46]
|
the Herring Monument
[34, 46]
|
Maria General Hospital
[35, 46]
|
Shalle Plaza
[36, 46]
|
Beaver Towers
[37, 46]
|
Clipper Avenue
[38, 46]
|
Crick Lane
[39, 46]
|
wasteland
[30, 47]
|
a cemetery
[31, 47]
|
Spurdell Walk Police Dept
[32, 47]
|
a warehouse
[33, 47]
|
Club Wistow
[34, 47]
|
Adey Plaza Fire Station
[35, 47]
|
Saltrow Alley Fire Station
[36, 47]
|
Gyllet Park
[37, 47]
|
Club Maule
[38, 47]
|
Brickenden Grove
[39, 47]
|
Vearncombe Alley Railway Station
[30, 48]
|
Cayme Drive
[31, 48]
|
a junkyard
[32, 48]
|
the Limbery Arms
[33, 48]
|
wasteland
[34, 48]
|
Floyer Boulevard
[35, 48]
|
Yapp Square Police Dept
[36, 48]
|
Ofield Drive
[37, 48]
|
Sheil Alley Railway Station
[38, 48]
|
Pask Square Railway Station
[39, 48]
|
the Gilling Museum
[30, 49]
|
St Eutychian's Hospital
[31, 49]
|
Fensome Street
[32, 49]
|
Lane Road
[33, 49]
|
Bousfield Alley
[34, 49]
|
Fothergill Towers
[35, 49]
|
Glisson Lane
[36, 49]
|
Vernoum Drive
[37, 49]
|
Attrell Avenue
[38, 49]
|
the Mules Monument
[39, 49]
|
|
Menu
City Map
Find Location
Route Finder
World View
Map Mirror
|


Test Automation Engineer with 8+ years of full system development and testing lifecycle experience, including implementing test automation from framework design to execution and test reporting.
Apart from being a QA, I enjoy most of my time working on my side projects. I love watching outdoor camping and bushcraft videos.
When forced indoors, I follow a number of sci-fi and adventure genre movies, television and web shows, I spend a large amount of my free time exploring the latest technology advancements in the test automation, front-end, and back-end web development world.