U ¯AdÕã @sVUdZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Zddl m Z m Z mZddlmZdd lmZdd lmZmZdd lmZe e¡ad Zd ZdZdZeZddgZ dddedƒe edƒgegdœZ!ee"d<ee!ƒZdZ#dZ$dZ%dZ&dddœZ'd d dœZ(d!d"gZ)d#d$d%d&d'd(gZ*dkd)d*„Z+e,eeee-dd+œd,d-„Z.d.d/„Z/d0d1„Z0dld2d3„Z1dmd4d5„Z2dnd6d7„Z3d8d9„Z4d:d;„Z5dd?„Z7d@dA„Z8dBdC„Z9dDdE„Z:dFdG„Z;dodIdJ„ZdqdOdP„Z?dQdR„Z@dSdT„ZAdUdV„ZBdWdX„ZCdYdZ„ZDd[d\„ZEd]d^„ZFd_d`„ZGdadb„ZHdrdcdd„ZIdedf„ZJdsdhdi„ZKdje4iZLdS)tz*Apt Configure: Configure apt for the user.éN)ÚLogger)Údedent)Úgpg)Úlog)ÚsubpÚ templaterÚutil)ÚCloud)ÚConfig)Ú MetaSchemaÚ get_meta_doc)Ú PER_INSTANCEz ^[\w-]+:\wz/etc/apt/trusted.gpgz/etc/apt/trusted.gpg.d/z/etc/apt/cloud-init.gpg.d/ZubuntuZdebianZcc_apt_configurez Apt ConfigurezConfigure apt for the useraä This module handles both configuration of apt options and adding source lists. There are configuration options such as ``apt_get_wrapper`` and ``apt_get_command`` that control how cloud-init invokes apt-get. These configuration options are handled on a per-distro basis, so consult documentation for cloud-init's distro support for instructions on using these config options. .. note:: To ensure that apt configuration is valid yaml, any strings containing special characters, especially ``:`` should be quoted. .. note:: For more information about apt configuration, see the ``Additional apt configuration`` example.a- apt: preserve_sources_list: false disable_suites: - $RELEASE-updates - backports - $RELEASE - mysuite primary: - arches: - amd64 - i386 - default uri: 'http://us.archive.ubuntu.com/ubuntu' search: - 'http://cool.but-sometimes-unreachable.com/ubuntu' - 'http://us.archive.ubuntu.com/ubuntu' search_dns: false - arches: - s390x - arm64 uri: 'http://archive-to-use-for-arm64.example.com/ubuntu' security: - arches: - default search_dns: true sources_list: | deb $MIRROR $RELEASE main restricted deb-src $MIRROR $RELEASE main restricted deb $PRIMARY $RELEASE universe restricted deb $SECURITY $RELEASE-security multiverse debconf_selections: set1: the-package the-package/some-flag boolean true conf: | APT { Get { Assume-Yes 'true'; Fix-Broken 'true'; } } proxy: 'http://[[user][:pass]@]host[:port]/' http_proxy: 'http://[[user][:pass]@]host[:port]/' ftp_proxy: 'ftp://[[user][:pass]@]host[:port]/' https_proxy: 'https://[[user][:pass]@]host[:port]/' sources: source1: keyid: 'keyid' keyserver: 'keyserverurl' source: 'deb [signed-by=$KEY_FILE] http:/// bionic main' source2: source: 'ppa:' source3: source: 'deb $MIRROR $RELEASE multiverse' key: | ------BEGIN PGP PUBLIC KEY BLOCK------- ------END PGP PUBLIC KEY BLOCK------- source4: source: 'deb $MIRROR $RELEASE multiverse' append: false key: | ------BEGIN PGP PUBLIC KEY BLOCK------- ------END PGP PUBLIC KEY BLOCK-------)ÚidÚnameÚtitleZ descriptionÚdistrosZexamplesÚ frequencyZactivate_by_schema_keysÚmetaz/var/lib/apt/listsz'/etc/apt/apt.conf.d/94cloud-init-configz)/etc/apt/apt.conf.d/90cloud-init-aptproxyzkeyserver.ubuntu.comz!http://archive.ubuntu.com/ubuntu/z"http://security.ubuntu.com/ubuntu/©ÚPRIMARYÚSECURITYz$http://ports.ubuntu.com/ubuntu-portsZamd64Zi386Zs390xZarm64ZarmhfZpowerpcZppc64elZriscv64cCsB|dkrt |¡}|tkr"t ¡S|tkr2t ¡Std|ƒ‚dS)z¢returns the default mirrors for the target. These depend on the architecture, for more see: https://wiki.ubuntu.com/UbuntuDevelopment/PackageArchive#PortsNz#No default mirror known for arch %s)rÚget_dpkg_architectureÚPRIMARY_ARCHESÚPRIMARY_ARCH_MIRRORSÚcopyÚ PORTS_ARCHESÚ PORTS_MIRRORSÚ ValueError)ÚarchÚtarget©r úC/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.pyÚget_default_mirrors s r")rÚcfgÚcloudrÚargsÚreturncCs\d}|dk r|at|ƒ}| di¡}t|tƒsBtdjt|ƒdƒ‚t||ƒt |||ƒdS)z•process the config for apt_config. This can be called from curthooks if a global apt config was provided or via the "apt" standalone command.NÚaptz9Expected dictionary for 'apt' config, found {config_type})Z config_type) ÚLOGÚconvert_to_v3_apt_formatÚgetÚ isinstanceÚdictrÚformatÚtypeÚapply_debconf_selectionsÚ apply_apt)rr#r$rr%rZapt_cfgr r r!Úhandle­s  ÿÿ r1cCs(t ¡r dSt d¡s$t d¡s$dSdS)N)Fzsystem is snappy.zapt-getr')Fzno apt commands.)TzApt is available.)rZsystem_is_snappyrZwhichr r r r!Ú_should_configure_on_empty_aptÇs r2c Cs*|s"tƒ\}}|s"t d|¡dSt d|¡tj|dd}t |¡}t|||d}t d|¡t | dd¡¡r˜t ||ƒt ||||ƒt |||ƒzt |t tƒWn"ttfk rÊt d ¡YnXd |kr&|}||d <|d |d <d} | d t¡} | rt | ¡j} t|d |||| ddS)Nz#Nothing to do: No apt config and %szhandling apt config: %s©rÚcodename)rzApt Mirror info: %sÚpreserve_sources_listFz)Failed to apply proxy or apt config info:ÚsourcesÚRELEASEÚMIRRORÚadd_apt_repo_match)rÚtemplate_paramsÚ aa_repo_match)r2r(ÚdebugrZ lsb_releaserÚfind_apt_mirror_infoZis_falser*Úadd_mirror_keysÚgenerate_sources_listÚrename_apt_listsÚapply_apt_configÚ APT_PROXY_FNÚ APT_CONFIG_FNÚIOErrorÚOSErrorÚ exceptionÚADD_APT_REPO_MATCHÚreÚcompileÚsearchÚadd_apt_sources) r#r$rZ should_configÚmsgÚreleaserÚmirrorsÚparamsZmatcherZmatchcfgr r r!r0ÐsB           ûr0cCs*| d¡s|d7}tjdg||dddS)Nó zdebconf-set-selectionsT©ÚdatarZcapture)Úendswithr)Ú selectionsrr r r!Údebconf_set_selectionsýs ürUcCs†g}g}|D]:}|tkr4sz,apply_debconf_selections..r3ú#z[:\s].*Úz pkgs_cfgd: %srzno need for reconfig)r*r(r<ÚjoinÚsortedÚkeysrUÚencodeÚsetÚitemsÚ splitlinesÚ startswithrHÚsubÚaddrZget_installed_packagesÚ intersectionrXr\) r#rrTZ pkgs_cfgdZ_keyZcontentÚliner[Zpkgs_installedZ need_reconfigr r`r!r/)s(          r/cCs6t t |d¡¡}t d|¡|D]}t |¡q"dS)z%clean out any local cloud-init configz/etc/cloud/cloud.cfg.d/*dpkg*z#cleaning cloud-init config from: %sN)ÚglobrÚ target_pathr(r<ÚosÚunlink)rZflistZdpkg_cfgr r r!Úclean_cloud_initLs  ÿ rtcCsL|}| d¡r|dd…}| d¡}|dkr<||dd…}| dd¡}|S)zæmirrorurl_to_apt_fileprefix Convert a mirror url to the file prefix used by apt on disk to store cache information for that mirror. To do so do: - take off ???:// - drop tailing / - convert in string / to _ú/réÿÿÿÿz://éNÚ_)rSÚfindÚreplace)ÚmirrorÚstringÚposr r r!Úmirrorurl_to_apt_fileprefixWs    r~c CsÞt|ƒ}t |t¡}| ¡D]¼\}}| |¡}|s4q|tjjt |ƒ}|tjjt |ƒ} || krfqt |ƒ} t   d|¡D]Z} d| | | d…f} t   d| | ¡zt | | ¡Wq|tk rÔt jdddYq|Xq|qdS)z>rename_apt_lists - rename apt lists to preserve old cache dataz%s_*z%s%sNzRenaming apt list %s to %szFailed to rename apt list:T)Úexc_info)r"rrqÚ APT_LISTSrir*rrÚpathÚsepr~rXrpr(r<ÚrenamerErY) Z new_mirrorsrrZdefault_mirrorsZprerZomirrorZnmirrorZoprefixZnprefixZolenÚfilenameZnewnamer r r!r@is$  r@cCs"||krt d||¡| ||¡S)z¥mirror_to_placeholder replace the specified mirror in a template with a placeholder string Checks for existance of the expected mirror and warns if not foundz%Expected mirror '%s' not found in: %s)r(rYrz)Útmplr{Z placeholderr r r!Úmirror_to_placeholder‚sr†cCs:ddddddœ}z ||}Wntk r4|}YnX|S)zÉthere are a few default names which will be auto-extended. This comes at the inability to use those names literally as suites, but on the other hand increases readability of the cfg quite a lotz$RELEASE-updatesz$RELEASE-backportsz$RELEASE-securityz$RELEASE-proposedz$RELEASE)ZupdatesZ backportsÚsecurityZproposedrM)ÚKeyError)ÚsuiteÚmappingZretsuiter r r!Úmap_known_suites‹sû  r‹c CsÖ|s|S|}|D]À}t|ƒ}t |d|i¡}t d||¡d}| d¡D]‚}| d¡r`||7}qH| ¡}t|ƒdkrÂd} |d d¡r®|dd …D]} | d7} |   d ¡r’q®q’|| |krÂd |}||7}qH|}q|S) zRreads the config for suites to be disabled and removes those from the templater7zDisabling suite %s as %srcTrbééú[Nú]z"# suite disabled by cloud-init: %s) r‹rÚ render_stringr(r<rjrkÚsplitrXrS) ÚdisabledÚsrcrMZretsrcr‰Z releasesuiteZnewsrcroZcolsZpcolÚcolr r r!Údisable_suitess2     r•cCs.dD]$}| |g¡D]}t|||dqqdS)z=Adds any keys included in the primary/security mirror clauses)Úprimaryr‡)Ú file_nameN)r*Ú add_apt_key)r#rr_r{r r r!r>Ãsr>c CsÀd}||dœ}|D] }||||<|||| ¡<q| dd¡}|dkrŽt d¡| d|jj¡}|sr| d¡}|s„t d¡dSt  |¡}t   ||¡} t | d ¡| |ƒ} tj || d d dS) zŽgenerate_sources_list create a source.list file based on a custom or default template by replacing mirrors and release in the templatez/etc/apt/sources.list)r7r4Ú sources_listNz1No custom template provided, fall back to builtinzsources.list.%sz sources.listz6No template found, not rendering /etc/apt/sources.listr•i¤)Úmode)Úlowerr*r(ÚinfoZget_template_filenameÚdistrorrYrZ load_filerrr•Ú write_file) r#rMrNr$ZaptsrcrOÚkr…Z template_fnZrenderedr’r r r!r?Ês,     ÿ ÿ  r?FcCsRt d|¡zt |¡j}td|||dWStjk rLt d¡‚YnXdS)zM actual adding of a key as defined in key argument to the system zAdding key: '%s'rm)Ú output_filerRÚhardenedz(failed to add apt GPG Key to apt keyringN) r(r<ÚpathlibÚPathZstemÚapt_keyrÚProcessExecutionErrorrF)r_r—r¡rrr r r!Úadd_apt_key_rawès   r¦cCs^d|kr8d|kr8t}d|kr$|d}t |d|¡|d<d|krZt|d|pR|d|dSdS)z› Add key to the system as defined in ent (if any). Supports raw keys or keyid's The latter will as a first step fetched to get the raw key Zkeyidr_Ú keyserverr„©r¡N)ÚDEFAULT_KEYSERVERrZ getkeybyidr¦)Úentrr¡r—r§r r r!r˜ös ÿr˜cCs|j ¡dS)N)rZupdate_package_sources)r$r r r!Úupdate_packages sr«c CsÌ|dkr i}|dkrtdƒ‚t|tƒs2td|ƒ‚|D]†}||}t d|¡d|kr`||d<d|krŒd|dkrŒt||dd }||d <n t||ƒ}d|kr q6|d}t ||¡}|d  d ¡sØt j   d |d¡|d<|d  d ¡sö|dd 7<||ƒr@ztjdd|g|dWq6tjk r<t d¡‚Yq6Xq6t ||d¡} z8d|} d} d|krv|dsvd} tj| | | dWq6tk r¼} zt d| | ¡‚W5d} ~ XYq6Xq6t|ƒdS)aÒ install keys and repo source .list files defined in 'sources' for each 'source' entry in the config: 1. expand template variables and write source .list file in /etc/apt/sources.list.d/ 2. install defined keys 3. update packages via distro-specific method (i.e. apt-key update) @param srcdict: a dict containing elements required @param cloud: cloud instance object Example srcdict value: { 'rio-grande-repo': { 'source': 'deb [signed-by=$KEY_FILE] $MIRROR $RELEASE main', 'keyid': 'B59D 5F15 97A5 04B7 E230 6DCA 0620 BBCF 0368 3F77', 'keyserver': 'pgp.mit.edu' } } Note: Deb822 format is not supported Nz did not get a valid repo matcherzunknown apt format: %szadding source/key '%s'r„Úsourcez $KEY_FILETr¨ZKEY_FILEruz/etc/apt/sources.list.d/z.listzadd-apt-repositoryz --no-updater3zadd-apt-repository failed.z%s ÚarWÚw)Úomodezfailed write to file %s: %s)rr+r,Ú TypeErrorr(r<r˜rrrkrrrrdrSrr¥rFrqrržrDr«) Úsrcdictr$rr:r;r„rªÚkey_filer¬ZsourcefnÚcontentsr¯Zdetailr r r!rK s`       ÿ þ  rKcCsxi}t d¡t|tƒr\t d¡|D]2}d|krHd|d<t |d¡}n|d}|||<q&nt|tƒrl|}ntdƒ‚|S)z1convert v1 apt format to v2 (dict in apt_sources)zIDEPRECATION: 'apt_sources' deprecated config key found. Use 'apt' insteadz9apt config: convert V1 to V2 format (source list to dict)r„zcloud_config_sources.listzunknown apt_sources format) r(rYr+rZr<rZ rand_dict_keyr,r)Zsrclistr±Zsrcentr_r r r!Úconvert_v1_to_v2_apt_formatfs ÿ    r´cCs,| |d¡dk r(| |¡||<||=dSdS)ziconvert an old key to the new one if the old one exists returns true if a key was found and convertedNTF©r*)ÚoldcfgÚaptcfgÚoldkeyÚnewkeyr r r!Ú convert_keys rºcCsJdddg}d}ddgi}|D]\}}t||||ƒrd}q|rF|g|d<d S) zBconvert old apt_mirror keys into the new more advanced mirror spec)Ú apt_mirrorÚuri)Úapt_mirror_searchrJ)Úapt_mirror_search_dnsÚ search_dnsFÚarchesÚdefaultTr–N)rº)r¶r·ZkeymapZ convertedZnewmcfgr¸r¹r r r!Úconvert_mirror‹sý  rÂc CsTddddddddddd d œ }g}|D]*}||kr$||d krD||=q$| |¡q$|sX|St d d  |¡¡| dd¡}|dk rèt d¡|D]V}||}||}||=|dksŒ| |d¡dkrÀqŒ|||krŒtd||||fƒ‚qŒ|Si}|D]"}||dk rðt|||||ƒqðt||ƒ|D]$}| |d¡dk r"td|ƒ‚q"||d<|S)z:convert old to new keys and adapt restructured mirror specr6NÚproxyÚ http_proxyÚ https_proxyÚ ftp_proxyr5r™r9) Ú apt_sourcesr»r½r¾Z apt_proxyZapt_http_proxyZ apt_ftp_proxyZapt_https_proxyZapt_preserve_sources_listZapt_custom_sources_listr9)NrczdDEPRECATION apt: converted deprecated config V2 to V3 format for keys '%s'. Use updated config keys.z, r'zODEPRECATION: apt config: deprecated V1/2 and V3 format specified, preferring V3z@Old and New apt format defined with unequal values %s vs %s @ %sz&old apt key '%s' left after conversion)rWr(rYrdr*rrºrÂ)r¶Z mapoldkeysZ needtoconvertr¸Z newaptcfgr¹Zverifyr·r r r!Úconvert_v2_to_v3_apt_formatshõ   ý ÿ  þÿ  rÈcCs,| dd¡}|dk r t|ƒ|d<t|ƒ}|S)z‰convert the old list based format to the new dict based one. After that convert the old dict keys/format to v3 a.k.a 'new apt config'rÇN)r*r´rÈ)r#rÇr r r!r)ès   r)c Cs¸d}|r´d}g}|dkrd}n|dkr,d}ntdƒ‚t ||¡j}d | d¡d d…¡}|rl| d |¡| d ¡g} |jj } d | |d | f} |D]} |  | | ¡q–t  | ¡}|S)zG Try to resolve a list of predefines DNS names to pick mirrors Nrcr–r{r‡zsecurity-mirrorzunknown mirror typeÚ.rŒz.%s)z .localdomainrczhttp://%s-%s%s/%sz%s) rrZget_hostname_fqdnÚfqdnrdr‘rWÚextendrrÚsearch_for_mirror) Z configuredÚ mirrortyper#r$r{ZmydomZdomsZ mirrordnsrÊZ mirror_listrZ mirrorfmtZpostr r r!Úsearch_for_mirror_dns÷s.ÿ rÎcCsX|dk r|dkr|}||dœS|j ¡}|rP| ¡}|d|d<|d|d<|St|ƒS)z^sets security mirror to primary if not defined. returns defaults if no mirrors are definedNrr–rr‡r)Z datasourceZget_package_mirror_inforr")ÚpmirrorÚsmirrorrr$Ú mirror_infoÚmr r r!Úupdate_mirror_info s    rÓcCsT| |d¡}|dkrdSd}|D].}| d¡p0g}||krB|Sd|kr |}q |S)zuout of a list of potential mirror configurations select and return the one matching the architecture (or default)NrÀrÁrµ)r#rÍrZmirror_cfg_listrÁZmirror_cfg_elemrÀr r r!Úget_arch_mirrorconfig8s rÔcCs`t|||ƒ}|dkrdS| dd¡}|dkr>t | dd¡¡}|dkr\t| dd¡|||ƒ}|S)z’pass the three potential stages of mirror specification returns None is neither of them found anything otherwise the first hit is returnedNr¼rJr¿)rÔr*rrÌrÎ)r#rÍrr$Zmcfgr{r r r!Ú get_mirrorKs   ÿrÕcCsn|dkrt ¡}t d|¡t|d||ƒ}t d|¡t|d||ƒ}t d|¡t||||ƒ}|d|d<|S) afind_apt_mirror_info find an apt_mirror given the cfg provided. It can check for separate config of primary and security mirrors If only primary is given security is assumed to be equal to primary If the generic apt_mirror is given that is defining for both Nz!got arch for mirror selection: %sr–zgot primary mirror: %sr‡zgot security mirror: %srr8)rrr(r<rÕrÓ)r#r$rrÏrÐrÑr r r!r=es    r=cs¶d}‡fdd„|Dƒ}t|ƒrBt d|¡t |d |¡d¡n"tj |¡rdt  |¡t d|¡ˆ  dd¡rt d |¡t |ˆ  d¡¡n"tj |¡r²t  |¡t d |¡dS) zHapply_apt_config Applies any apt*proxy config from if specified ))rÃúAcquire::http::Proxy "%s";)rÄrÖ)rÆzAcquire::ftp::Proxy "%s";)rÅzAcquire::https::Proxy "%s";cs(g|] \}}ˆ |¡r|ˆ |¡‘qSr rµ)r^rZfmt©r#r r!ra‰s z$apply_apt_config..zwrite apt proxy info to %sr]z#no apt proxy configured, removed %sÚconfNzwrite apt config info to %sz$no apt config configured, removed %s) rXr(r<rržrdrrrÚisfileZdel_filer*)r#Z proxy_fnameZ config_fnameZcfgsZproxiesr r×r!rA}s        rATcsVdd„‰‡‡‡fdd„}‡‡fdd„}|dkr4|ƒS|dksD|d krJ|ƒStd ƒ‚d S) adapt-key replacement commands implemented: 'add', 'list', 'finger' @param output_file: name of output gpg file (without .gpg or .asc) @param data: key contents @param human_output: list keys formatted for human parsing @param hardened: write keys to to /etc/apt/cloud-init.gpg.d/ (referred to with [signed-by] in sources file) cSsTtj t¡rtgng}t t¡D]&}| d¡s8| d¡r | t|¡q |rP|SdS)zÎreturn all apt keys /etc/apt/trusted.gpg (if it exists) and all keyfiles (and symlinks to keyfiles) in /etc/apt/trusted.gpg.d/ are returned based on apt-key implementation z.gpgz.ascrc)rrrrÙÚAPT_LOCAL_KEYSÚlistdirÚAPT_TRUSTED_GPG_DIRrSrW)Z key_filesÚfiler r r!Ú_get_key_files§s zapt_key.._get_key_filesc s d}ˆst td ˆ¡¡n€z2ˆr&tnt}t ˆ¡}d |ˆ¡}t ||¡WnLt j k rvt td ˆ¡¡Yn&t k ršt td ˆ¡¡YnX|S)ziapt-key add returns filepath to new keyring, or '/dev/null' when an error occurs z /dev/nullz)Unknown filename, failed to add key: "{}"z{}{}.gpgz Gpg error, failed to add key: {}z#Decode error, failed to add key: {}) rZlogexcr(r-ÚCLOUD_INIT_GPG_DIRrÜrZdearmorržrr¥ÚUnicodeDecodeError)r—Zkey_dirÚstdout)rRr¡r r r!Ú apt_key_add¶s.ÿ ÿ  ÿÿ zapt_key..apt_key_addc sfg}ˆƒD]P}z| tj|ˆd¡Wq tjk rX}zt d||¡W5d}~XYq Xq d |¡S)zapt-key list returns string of all trusted keys (in /etc/apt/trusted.gpg and /etc/apt/trusted.gpg.d/) )Ú human_outputzFailed to list key "%s": %sNr])rWrrZrr¥r(rYrd)Zkey_listr²Úerror)rÞrãr r!Ú apt_key_listÒs "zapt_key..apt_key_listrmZfingerrZz@apt_key() commands add, list, and finger are currently supportedN)r)Zcommandr rRr¡rãrârår )rÞrRr¡rãr r!r¤™sÿr¤z cloud-init)NN)N)N)N)FN)NFN)NNN)N)NNFT)MÚ__doc__rprrr¢rHZloggingrÚtextwraprZ cloudinitrrrrrZcloudinit.cloudr Zcloudinit.configr Zcloudinit.config.schemar r Zcloudinit.settingsr Z getLoggerÚ__name__r(rGrÚrÜrßrrrÚ__annotations__r€rCrBr©rrrrr"ÚstrrZr1r2r0rUr\r/rtr~r@r†r‹r•r>r?r¦r˜r«rKr´rºrÂrÈr)rÎrÓrÔrÕr=rAr¤rVr r r r!Ú s¼        ÿÿÿE£`þþ þ  - ! #  &  ÿ Y K) ÿ Rÿ