Hi Jayaraj,

Yes, you need to generate a new manifest for each upgrade of a CF version. Properties might be added, jobs may change, new resource pools may be required, older ones may be removed, etc between versions. So for each version upgrade you need to regenerate the manifest. 

One way a lot of CloudFoundry teams manage this is by having a stub file which contains properties and customizations specific to their CF installation. This stub gets merged with other CF templates to generate the actual manifest yml file.

To see a good example of this look at the bosh-lite stub and the template generation code:

https://github.com/cloudfoundry/cf-release/tree/master/bosh-lite/stubs
https://github.com/cloudfoundry/cf-release/blob/master/scripts/generate-bosh-lite-dev-manifest

For other serious installations (AWS, OpenStack, vSphere, etc) we have the generate_deployment_manifest script which lets you specify the IaaS and then the path to your stub. This is the recommended way of generating your manifest and you should re-generate the manifest for each new release version.

Rohit

On Tue, Feb 16, 2016 at 6:31 PM, Jayarajan Ramapurath Kozhummal (jayark) <jayark@cisco.com> wrote:
Hi Rohit,

We did not generate a fresh manifest for the CF version 230. We had a manifest file which is used for deploying CF version 205.
Do we need to generate the manifest file for each CF release?  Can we generate the new manifest file for CF version 230 using scripts/generate_deployment_manifest and copy paste the missing contents to our deployment manifest file?

Thanks
Jayaraj

From: Rohit Kumar <rokumar@pivotal.io>
Date: Tuesday, February 16, 2016 at 5:25 PM

To: Jayarajan Ramapurath Kozhummal <jayark@cisco.com>
Cc: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@lists.cloudfoundry.org>, "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase@cisco.com>
Subject: Re: [cf-dev] Reg cant find template : metron_agent

Those properties are used to specify the etcd machines to loggregator. Those typically get auto-filled by spiff and you don't need to specify them explicitly in the properties section [1]. Did you not generate your manifest with the help of `scripts/generate_deployment_manifest` script in cf-release? 

Rohit


On Tue, Feb 16, 2016 at 5:55 PM, Jayarajan Ramapurath Kozhummal (jayark) <jayark@cisco.com> wrote:
Hi Rohit,

I added the cloud_controller IP for the below property and I am running into the below exception now:-

  Started preparing configuration > Binding configuration. Failed: Error filling in template `etcd_bosh_utils.sh.erb' for `cloud_controller/0' (line 31: Can't find property `["etcd.cluster"]') (00:00:01)


Error 100: Error filling in template `etcd_bosh_utils.sh.erb' for `cloud_controller/0' (line 31: Can't find property `["etcd.cluster"]')


Task 55 error


For a more detailed error report, run: bosh task 55 —debug



Is there a reference which I can use to find what values to be filled for the missing properties?


Thanks

Jayaraj



From: Jayarajan Ramapurath Kozhummal <jayark@cisco.com>
Date: Tuesday, February 16, 2016 at 4:29 PM

To: Rohit Kumar <rokumar@pivotal.io>
Cc: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@lists.cloudfoundry.org>, "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase@cisco.com>
Subject: Re: [cf-dev] Reg cant find template : metron_agent

Thanks  Rohit!

That worked for metro_agent issue. Now running into the following issue after filling the property you mentioned.  

  Started preparing configuration > Binding configuration. Failed: Error filling in template `metron_agent.json.erb' for `nfs/0' (line 7: Can't find property `["loggregator.etcd.machines"]') (00:00:00)


Error 100: Error filling in template `metron_agent.json.erb' for `nfs/0' (line 7: Can't find property `["loggregator.etcd.machines"]')


Task 54 error


For a more detailed error report, run: bosh task 54 —debug



Nithiya was also running into the same issues. The error posted is after applying some workarounds as seen in the web.


Regards

Jayaraj



From: Rohit Kumar <rokumar@pivotal.io>
Date: Tuesday, February 16, 2016 at 3:53 PM
To: Jayarajan Ramapurath Kozhummal <jayark@cisco.com>
Cc: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@lists.cloudfoundry.org>, "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase@cisco.com>
Subject: Re: [cf-dev] Reg cant find template : metron_agent

OK cool. The error which you are getting now is different from what you had originally posted. You need to include the following property in your deployment and it should get fixed:

properties:
  metron_agent:
    deployment: <name of your deployment> 


On Tue, Feb 16, 2016 at 3:41 PM, Jayarajan Ramapurath Kozhummal (jayark) <jayark@cisco.com> wrote:
Hi Rohit,

Please see the output:-

root@vms-inception-vm-2:/opt/cisco/vms-installer/tenant-vmsdev5control/cf-deploy# bosh -n deploy

Acting as user 'admin' on deployment 'cf-vmsdev5control' on 'vms-installdev5-control-66380'

Getting deployment properties from director...

Unable to get properties list from director, trying without it...

Cannot get current deployment information from director, possibly a new deployment


Deploying

---------


Director task 51

  Started unknown

  Started unknown > Binding deployment. Done (00:00:00)


  Started preparing deployment

  Started preparing deployment > Binding releases. Done (00:00:00)

  Started preparing deployment > Binding existing deployment. Done (00:00:00)

  Started preparing deployment > Binding resource pools. Done (00:00:00)

  Started preparing deployment > Binding stemcells. Done (00:00:00)

  Started preparing deployment > Binding templates. Done (00:00:00)

  Started preparing deployment > Binding properties. Done (00:00:00)

  Started preparing deployment > Binding unallocated VMs. Done (00:00:01)

  Started preparing deployment > Binding instance networks. Done (00:00:00)


  Started preparing package compilation > Finding packages to compile. Done (00:00:00)


  Started compiling packages

  Started compiling packages > rootfs_cflinuxfs2/3232d35298f26bcfb153d964e329fcb42c77051f

  Started compiling packages > haproxy/f5d89b125a66892628a8cd61d23be7f9b0d31171

  Started compiling packages > uaa/0e15122de61644748d111b619aff4487726f8378

  Started compiling packages > golang1.5/ef3267f8998cebcdc86a477126e79e465753aaf1

  Started compiling packages > uaa_utils/8ee843cd3e50520398f28541c513ac0d16b00877

  Started compiling packages > postgres-9.4.5/06a51985e0701707b27d45c7a5757171b5cefb8c

  Started compiling packages > buildpack_binary/e0c8736b073d83c2459519851b5736c288311d92

  Started compiling packages > buildpack_staticfile/47c22ec219ca96215c509572f7a59aae55e45535

  Started compiling packages > buildpack_php/6dae2301648646cd8ed544af53ff34be0497efe0

  Started compiling packages > buildpack_python/a5d5eeb5e255ceb3282424a28c74a4bccd3316e9

     Done compiling packages > uaa/0e15122de61644748d111b619aff4487726f8378 (00:02:52)

  Started compiling packages > buildpack_go/08a35c7097417bedf06812c7ac8931d950dfae60

     Done compiling packages > buildpack_php/6dae2301648646cd8ed544af53ff34be0497efe0 (00:03:02)

  Started compiling packages > buildpack_nodejs/da88c1de3e899a27d33c5a8d6e08e151b42a1aa8. Done (00:00:05)

  Started compiling packages > buildpack_ruby/d37b44b37b7c95077fd9698879b78561ac0aaf52

     Done compiling packages > buildpack_go/08a35c7097417bedf06812c7ac8931d950dfae60 (00:00:37)

  Started compiling packages > buildpack_java_offline/f6b99f87508400e9d75926c1546e8d08177072ef

     Done compiling packages > buildpack_ruby/d37b44b37b7c95077fd9698879b78561ac0aaf52 (00:00:26)

  Started compiling packages > buildpack_java/0dd2a9074cdfee66f56d6a9e958c2b9e1fa9337c. Done (00:00:02)

  Started compiling packages > nginx/bf3af6163e13887aacd230bbbc5eff90213ac6af

     Done compiling packages > buildpack_java_offline/f6b99f87508400e9d75926c1546e8d08177072ef (00:00:22)

  Started compiling packages > ruby-2.2.4/dd1b827e6ea0ca7e9fcb95d08ae81fb82f035261

     Done compiling packages > nginx/bf3af6163e13887aacd230bbbc5eff90213ac6af (00:00:33)

  Started compiling packages > libpq/14d0b1290ea238243d04dd46d1a9635e6e9812bb

     Done compiling packages > buildpack_python/a5d5eeb5e255ceb3282424a28c74a4bccd3316e9 (00:04:10)

  Started compiling packages > libmariadb/dcc142dd0798ae557193f08bc46e9bdd97e4c6f3. Done (00:00:02)

  Started compiling packages > ruby-2.1.8/b5bf6af82bae947ad255e426001308acfc2244ee

     Done compiling packages > uaa_utils/8ee843cd3e50520398f28541c513ac0d16b00877 (00:04:25)

  Started compiling packages > etcd-common/a5492fb0ad41a80d2fa083172c0430073213a296

     Done compiling packages > libpq/14d0b1290ea238243d04dd46d1a9635e6e9812bb (00:00:18)

  Started compiling packages > golang1.4/714698bc352d2a1dbe321376f0676037568147bb

     Done compiling packages > etcd-common/a5492fb0ad41a80d2fa083172c0430073213a296 (00:00:02)

  Started compiling packages > loggregator_common/e401816a4748292163679fafcbd8f818ed8154a5

     Done compiling packages > haproxy/f5d89b125a66892628a8cd61d23be7f9b0d31171 (00:04:28)

  Started compiling packages > debian_nfs_server/aac05f22582b2f9faa6840da056084ed15772594

     Done compiling packages > loggregator_common/e401816a4748292163679fafcbd8f818ed8154a5 (00:00:03)

  Started compiling packages > common/e401816a4748292163679fafcbd8f818ed8154a5

     Done compiling packages > debian_nfs_server/aac05f22582b2f9faa6840da056084ed15772594 (00:00:04)

     Done compiling packages > common/e401816a4748292163679fafcbd8f818ed8154a5 (00:00:02)

     Done compiling packages > golang1.4/714698bc352d2a1dbe321376f0676037568147bb (00:00:15)

  Started compiling packages > dea_logging_agent/3179906f4e18fa39bf8baa60c92ee51fb7ce4e22

  Started compiling packages > loggregator_trafficcontroller/612624b9a615310d1d87053101c0f64b87038ab5

  Started compiling packages > doppler/4abad345222d75f714fc3b7524c87b1829dcd187

     Done compiling packages > dea_logging_agent/3179906f4e18fa39bf8baa60c92ee51fb7ce4e22 (00:00:09)

  Started compiling packages > gnatsd/0242557ff8fc93c42ff54aa642c524b17ce203eb

     Done compiling packages > buildpack_staticfile/47c22ec219ca96215c509572f7a59aae55e45535 (00:04:51)

  Started compiling packages > etcd_metrics_server/fc0f1835cd8e95ca86cf3851645486531ae4f12b

     Done compiling packages > loggregator_trafficcontroller/612624b9a615310d1d87053101c0f64b87038ab5 (00:00:14)

  Started compiling packages > etcd/d43feb5cdad0809d109df0afe6cd3c315dc94a61

     Done compiling packages > doppler/4abad345222d75f714fc3b7524c87b1829dcd187 (00:00:17)

  Started compiling packages > metron_agent/4dfd17660ea7654bcdfbb81a15cef3b86ac22aab

     Done compiling packages > gnatsd/0242557ff8fc93c42ff54aa642c524b17ce203eb (00:00:08)

     Done compiling packages > etcd_metrics_server/fc0f1835cd8e95ca86cf3851645486531ae4f12b (00:00:11)

     Done compiling packages > metron_agent/4dfd17660ea7654bcdfbb81a15cef3b86ac22aab (00:00:14)

     Done compiling packages > etcd/d43feb5cdad0809d109df0afe6cd3c315dc94a61 (00:00:30)

     Done compiling packages > rootfs_cflinuxfs2/3232d35298f26bcfb153d964e329fcb42c77051f (00:05:47)

     Done compiling packages > buildpack_binary/e0c8736b073d83c2459519851b5736c288311d92 (00:07:34)

     Done compiling packages > golang1.5/ef3267f8998cebcdc86a477126e79e465753aaf1 (00:07:38)

  Started compiling packages > gorouter/cbbf5f8f71a32cf205d910fe86ef3e5eaa1897f5

  Started compiling packages > hm9000/082bbefc4bf586e9195ce94d21dfc4a1e7c6798f

     Done compiling packages > ruby-2.2.4/dd1b827e6ea0ca7e9fcb95d08ae81fb82f035261 (00:03:58)

  Started compiling packages > dea_next/6193e865f0a87f054d550f0e8c6ff3173e216e0e

  Started compiling packages > warden/0fc9616fdc0263f6093a58d9d4da5bb47e337ec2

  Started compiling packages > nginx_newrelic_plugin/3bf72c30bcda79a44863a2d1a6f932fe0a5486a5

  Started compiling packages > cloud_controller_ng/9ca58fcb7c289431af16f161078d22ada352ff20

     Done compiling packages > nginx_newrelic_plugin/3bf72c30bcda79a44863a2d1a6f932fe0a5486a5 (00:00:12)

     Done compiling packages > gorouter/cbbf5f8f71a32cf205d910fe86ef3e5eaa1897f5 (00:00:29)

     Done compiling packages > hm9000/082bbefc4bf586e9195ce94d21dfc4a1e7c6798f (00:00:29)

     Done compiling packages > ruby-2.1.8/b5bf6af82bae947ad255e426001308acfc2244ee (00:04:00)

  Started compiling packages > collector/9f8dfbcbcfffb124820327ad2ad4fee35e51d236

  Started compiling packages > nats/2230720d1021af6c2c90cd7f3983264ab351043b

     Done compiling packages > warden/0fc9616fdc0263f6093a58d9d4da5bb47e337ec2 (00:00:36)

     Done compiling packages > nats/2230720d1021af6c2c90cd7f3983264ab351043b (00:00:23)

     Done compiling packages > collector/9f8dfbcbcfffb124820327ad2ad4fee35e51d236 (00:00:39)

     Done compiling packages > dea_next/6193e865f0a87f054d550f0e8c6ff3173e216e0e (00:01:51)

     Done compiling packages > postgres-9.4.5/06a51985e0701707b27d45c7a5757171b5cefb8c (00:09:43)

     Done compiling packages > cloud_controller_ng/9ca58fcb7c289431af16f161078d22ada352ff20 (00:03:09)

     Done compiling packages (00:10:58)


  Started preparing dns > Binding DNS. Done (00:00:00)


  Started creating bound missing vms

  Started creating bound missing vms > small/0

  Started creating bound missing vms > small/1

  Started creating bound missing vms > small/2

  Started creating bound missing vms > medium/0

  Started creating bound missing vms > medium/1

  Started creating bound missing vms > medium/2

  Started creating bound missing vms > medium/3

  Started creating bound missing vms > medium/4

  Started creating bound missing vms > medium/5

  Started creating bound missing vms > large/0

  Started creating bound missing vms > large/1

  Started creating bound missing vms > large/2

  Started creating bound missing vms > large/3

  Started creating bound missing vms > large/4

  Started creating bound missing vms > large/5

  Started creating bound missing vms > large/6

  Started creating bound missing vms > large/7

  Started creating bound missing vms > xlarge/0

  Started creating bound missing vms > xlarge/1

     Done creating bound missing vms > medium/2 (00:01:53)

     Done creating bound missing vms > large/4 (00:01:55)

     Done creating bound missing vms > medium/3 (00:01:56)

     Done creating bound missing vms > xlarge/0 (00:01:56)

     Done creating bound missing vms > large/1 (00:01:59)

     Done creating bound missing vms > medium/0 (00:02:02)

     Done creating bound missing vms > medium/4 (00:02:05)

     Done creating bound missing vms > large/6 (00:02:18)

     Done creating bound missing vms > large/2 (00:02:20)

     Done creating bound missing vms > large/3 (00:02:20)

     Done creating bound missing vms > large/5 (00:02:20)

     Done creating bound missing vms > medium/5 (00:02:25)

     Done creating bound missing vms > xlarge/1 (00:02:29)

     Done creating bound missing vms > large/7 (00:02:31)

     Done creating bound missing vms > medium/1 (00:02:33)

     Done creating bound missing vms > large/0 (00:02:43)

     Done creating bound missing vms > small/2 (00:02:51)

     Done creating bound missing vms > small/1 (00:03:25)

     Done creating bound missing vms > small/0 (00:03:31)

     Done creating bound missing vms (00:03:31)


  Started binding instance vms

  Started binding instance vms > nfs/0

  Started binding instance vms > cloud_controller/0

  Started binding instance vms > loggregator/0

  Started binding instance vms > loggregator_trafficcontroller/0

  Started binding instance vms > api_worker/0

  Started binding instance vms > dea-spare/0

  Started binding instance vms > dea-spare/1

  Started binding instance vms > router/0

  Started binding instance vms > router/1

  Started binding instance vms > haproxy/0

  Started binding instance vms > cassandra/0

  Started binding instance vms > cassandra_seed/0

  Started binding instance vms > zookeeper/0

  Started binding instance vms > redis/0

  Started binding instance vms > zookeeper/1

  Started binding instance vms > zookeeper/2

  Started binding instance vms > kafka/1

  Started binding instance vms > kafka/0

  Started binding instance vms > kafka/2

     Done binding instance vms > loggregator_trafficcontroller/0 (00:00:00)

     Done binding instance vms > haproxy/0 (00:00:01)

     Done binding instance vms > router/1 (00:00:01)

     Done binding instance vms > cassandra/0 (00:00:01)

     Done binding instance vms > api_worker/0 (00:00:01)

     Done binding instance vms > loggregator/0 (00:00:01)

     Done binding instance vms > dea-spare/1 (00:00:01)

     Done binding instance vms > zookeeper/0 (00:00:01)

     Done binding instance vms > cassandra_seed/0 (00:00:01)

     Done binding instance vms > kafka/1 (00:00:01)

     Done binding instance vms > zookeeper/1 (00:00:01)

     Done binding instance vms > cloud_controller/0 (00:00:01)

     Done binding instance vms > nfs/0 (00:00:01)

     Done binding instance vms > router/0 (00:00:02)

     Done binding instance vms > zookeeper/2 (00:00:02)

     Done binding instance vms > redis/0 (00:00:02)

     Done binding instance vms > kafka/0 (00:00:02)

     Done binding instance vms > dea-spare/0 (00:00:02)

     Done binding instance vms > kafka/2 (00:00:02)

     Done binding instance vms (00:00:02)


  Started preparing configuration > Binding configuration. Failed: Error filling in template `metron_agent.json.erb' for `nfs/0' (line 5: Can't find property `["metron_agent.deployment"]') (00:00:00)


Error 100: Error filling in template `metron_agent.json.erb' for `nfs/0' (line 5: Can't find property `["metron_agent.deployment"]')


Task 51 error


For a more detailed error report, run: bosh task 51 --debug


Thanks
Jayaraj

From: Jayarajan Ramapurath Kozhummal <jayark@cisco.com>
Date: Tuesday, February 16, 2016 at 2:29 PM
To: Rohit Kumar <rokumar@pivotal.io>
Cc: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@lists.cloudfoundry.org>, "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase@cisco.com>

Subject: Re: [cf-dev] Reg cant find template : metron_agent

Hi Rohit,

Following steps followed :-
  • git clone https://github.com/cloudfoundry/cf-release.git
  • root@vms-inception-vm-2:/opt/cisco/vms-installer/cf-release/scripts# ./update 

  • root@vms-inception-vm-2:/opt/cisco/vms-installer/cf-release# bosh create release releases/cf-230.yml --with-tarball

  • root@vms-inception-vm-2:/opt/cisco/vms-installer/cf-release/releases# bosh upload release cf-230.tgz 

  • root@vms-inception-vm-2:/opt/cisco/vms-installer/tenant-vmsdev5control/cf-deploy# bosh -n deployment cf-vmsdev5control.yml

Deployment set to `/opt/cisco/vms-installer/tenant-vmsdev5control/cf-deploy/cf-vmsdev5control.yml

  • root@vms-inception-vm-2:/opt/cisco/vms-installer/tenant-vmsdev5control/cf-deploy# bosh -n deploy


I have shared the sample deployment yml file we are using for your reference.



Thanks

Jayaraj


From: Rohit Kumar <rokumar@pivotal.io>
Date: Tuesday, February 16, 2016 at 2:15 PM
To: Jayarajan Ramapurath Kozhummal <jayark@cisco.com>
Cc: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@lists.cloudfoundry.org>
Subject: Re: [cf-dev] Reg cant find template : metron_agent

Can you also list the commands on how you are creating, uploading and deploying the release?

On Tue, Feb 16, 2016 at 2:42 PM, Jayarajan Ramapurath Kozhummal (jayark) <jayark@cisco.com> wrote:

Thanks a lot Rohit for replying back very quickly!
I have run the scripts/update command after cloning the cf-release GIT repo.
Please see the command output below:-

root@automation-vm-jayark:/opt/cisco/vms-installer/cf-release/src/loggregator# find . -type d -maxdepth 2

find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it).  Please specify options before other arguments.


.

./src

./src/doppler

./src/loggregator

./src/trafficcontroller

./src/syslog_drain_binder

./src/bitbucket.org

./src/monitor

./src/matchers

./src/signalmanager

./src/deaagent

./src/tools

./src/truncatingbuffer

./src/profiler

./src/logger

./src/lats

./src/common

./src/integration_tests

./src/metron

./src/github.com

./packages

./packages/doppler

./packages/loggregator_trafficcontroller

./packages/syslog_drain_binder

./packages/dea_logging_agent

./packages/loggregator_common

./packages/loggregator-acceptance-tests

./packages/golang1.4

./packages/metron_agent

./docs

./config

./jobs

./jobs/doppler

./jobs/loggregator_trafficcontroller

./jobs/syslog_drain_binder

./jobs/dea_logging_agent

./jobs/loggregator-acceptance-tests

./jobs/metron_agent

./bin

./git-hooks

./samples


Thanks
Jayaraj

From: Rohit Kumar <rokumar@pivotal.io>
Date: Tuesday, February 16, 2016 at 9:30 AM
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@lists.cloudfoundry.org>
Cc: Jayarajan Ramapurath Kozhummal <jayark@cisco.com>
Subject: Re: [cf-dev] Reg cant find template : metron_agent

Did you make sure to run "scripts/update" after cloning the cf-release repo? Can you run "find . -type d -maxdepth 2" from within the "src/loggregator" directory in cf-release and reply with what you get as output?

Rohit

On Tue, Feb 16, 2016 at 1:39 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase@cisco.com> wrote:

Hi

 

I am working on cloud foundry and I could create a development bosh release of cloud foundry using the following source:

git clone https://github.com/cloudfoundry/cf-release.git (added some rules  in haproxy.conf file)

 

When I tried to deploy with the dev release of cloud-foundry, I get the following error:

 

Started preparing deployment

  Started preparing deployment > Binding releases. Done (00:00:00)

  Started preparing deployment > Binding existing deployment. Done (00:00:00)

  Started preparing deployment > Binding resource pools. Done (00:00:00)

  Started preparing deployment > Binding stemcells. Done (00:00:00)

  Started preparing deployment > Binding templates. Failed: Can't find template `metron_agent' (00:00:00)

 

Error 190012: Can't find template `metron_agent'

 

Kindly help me figure out the issue of the error, as this is a show-stopper for us..

 

Regards

Nithiaysri